{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线性回归应用案例\n",
    "前面我们通过一个小例子介绍了线性回归模型。下面我们用一个现实的数据集来应用线性回归算法。假如你去参加聚会，想喝最好的红酒，可以让朋友推荐，不过你觉得他们也不靠谱。作为科学控，你带了pH试纸和一堆测量工具来测酒的理化性质，然后选一个最好的，周围的小伙伴都无语了，你亮瞎了世界。\n",
    "\n",
    "网上有相关的酒数据集可以参考，UCI机器学习项目的酒数据集收集了1599种酒的测试数据,数据下载链接[在此](http://www3.dsi.uminho.pt/pcortez/wine/winequality.zip)。收集完数据自然要用线性回归来研究一下，响应变量是0-10的整数值，我们也可以把这个问题看成是一个分类问题。不过本章还是把相应变量作为连续值来处理。\n",
    "\n",
    "### 探索数据\n",
    "scikit-learn作为机器学习系统，其探索数据的能力是不能与SPSS和R语言相媲美的。不过我们有Pandas库，可以方便的读取数据，完成描述性统计工作。我们通过描述性统计来设计模型。Pandas引入了R语言的dataframe，一种二维表格式异质（heterogeneous）数据结构。Pandas更多功能请见[文档](http://pandas.pydata.org)，这里只用一部分功能，都很容易使用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，我们读取.csv文件生成dataframe："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.4</td>\n",
       "      <td>0.70</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.076</td>\n",
       "      <td>11.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>0.9978</td>\n",
       "      <td>3.51</td>\n",
       "      <td>0.56</td>\n",
       "      <td>9.4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.8</td>\n",
       "      <td>0.88</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2.6</td>\n",
       "      <td>0.098</td>\n",
       "      <td>25.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>0.9968</td>\n",
       "      <td>3.20</td>\n",
       "      <td>0.68</td>\n",
       "      <td>9.8</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.8</td>\n",
       "      <td>0.76</td>\n",
       "      <td>0.04</td>\n",
       "      <td>2.3</td>\n",
       "      <td>0.092</td>\n",
       "      <td>15.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.9970</td>\n",
       "      <td>3.26</td>\n",
       "      <td>0.65</td>\n",
       "      <td>9.8</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>11.2</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.56</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.075</td>\n",
       "      <td>17.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.9980</td>\n",
       "      <td>3.16</td>\n",
       "      <td>0.58</td>\n",
       "      <td>9.8</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7.4</td>\n",
       "      <td>0.70</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.076</td>\n",
       "      <td>11.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>0.9978</td>\n",
       "      <td>3.51</td>\n",
       "      <td>0.56</td>\n",
       "      <td>9.4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "0            7.4              0.70         0.00             1.9      0.076   \n",
       "1            7.8              0.88         0.00             2.6      0.098   \n",
       "2            7.8              0.76         0.04             2.3      0.092   \n",
       "3           11.2              0.28         0.56             1.9      0.075   \n",
       "4            7.4              0.70         0.00             1.9      0.076   \n",
       "\n",
       "   free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
       "0                 11.0                  34.0   0.9978  3.51       0.56   \n",
       "1                 25.0                  67.0   0.9968  3.20       0.68   \n",
       "2                 15.0                  54.0   0.9970  3.26       0.65   \n",
       "3                 17.0                  60.0   0.9980  3.16       0.58   \n",
       "4                 11.0                  34.0   0.9978  3.51       0.56   \n",
       "\n",
       "   alcohol  quality  \n",
       "0      9.4        5  \n",
       "1      9.8        5  \n",
       "2      9.8        5  \n",
       "3      9.8        6  \n",
       "4      9.4        5  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv('mlslpic/winequality-red.csv', sep=';')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "      <td>1599.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>8.319637</td>\n",
       "      <td>0.527821</td>\n",
       "      <td>0.270976</td>\n",
       "      <td>2.538806</td>\n",
       "      <td>0.087467</td>\n",
       "      <td>15.874922</td>\n",
       "      <td>46.467792</td>\n",
       "      <td>0.996747</td>\n",
       "      <td>3.311113</td>\n",
       "      <td>0.658149</td>\n",
       "      <td>10.422983</td>\n",
       "      <td>5.636023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.741096</td>\n",
       "      <td>0.179060</td>\n",
       "      <td>0.194801</td>\n",
       "      <td>1.409928</td>\n",
       "      <td>0.047065</td>\n",
       "      <td>10.460157</td>\n",
       "      <td>32.895324</td>\n",
       "      <td>0.001887</td>\n",
       "      <td>0.154386</td>\n",
       "      <td>0.169507</td>\n",
       "      <td>1.065668</td>\n",
       "      <td>0.807569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>4.600000</td>\n",
       "      <td>0.120000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.900000</td>\n",
       "      <td>0.012000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>0.990070</td>\n",
       "      <td>2.740000</td>\n",
       "      <td>0.330000</td>\n",
       "      <td>8.400000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>7.100000</td>\n",
       "      <td>0.390000</td>\n",
       "      <td>0.090000</td>\n",
       "      <td>1.900000</td>\n",
       "      <td>0.070000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0.995600</td>\n",
       "      <td>3.210000</td>\n",
       "      <td>0.550000</td>\n",
       "      <td>9.500000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>7.900000</td>\n",
       "      <td>0.520000</td>\n",
       "      <td>0.260000</td>\n",
       "      <td>2.200000</td>\n",
       "      <td>0.079000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>0.996750</td>\n",
       "      <td>3.310000</td>\n",
       "      <td>0.620000</td>\n",
       "      <td>10.200000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>9.200000</td>\n",
       "      <td>0.640000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>2.600000</td>\n",
       "      <td>0.090000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>0.997835</td>\n",
       "      <td>3.400000</td>\n",
       "      <td>0.730000</td>\n",
       "      <td>11.100000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>15.900000</td>\n",
       "      <td>1.580000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>15.500000</td>\n",
       "      <td>0.611000</td>\n",
       "      <td>72.000000</td>\n",
       "      <td>289.000000</td>\n",
       "      <td>1.003690</td>\n",
       "      <td>4.010000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>14.900000</td>\n",
       "      <td>8.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       fixed acidity  volatile acidity  citric acid  residual sugar  \\\n",
       "count    1599.000000       1599.000000  1599.000000     1599.000000   \n",
       "mean        8.319637          0.527821     0.270976        2.538806   \n",
       "std         1.741096          0.179060     0.194801        1.409928   \n",
       "min         4.600000          0.120000     0.000000        0.900000   \n",
       "25%         7.100000          0.390000     0.090000        1.900000   \n",
       "50%         7.900000          0.520000     0.260000        2.200000   \n",
       "75%         9.200000          0.640000     0.420000        2.600000   \n",
       "max        15.900000          1.580000     1.000000       15.500000   \n",
       "\n",
       "         chlorides  free sulfur dioxide  total sulfur dioxide      density  \\\n",
       "count  1599.000000          1599.000000           1599.000000  1599.000000   \n",
       "mean      0.087467            15.874922             46.467792     0.996747   \n",
       "std       0.047065            10.460157             32.895324     0.001887   \n",
       "min       0.012000             1.000000              6.000000     0.990070   \n",
       "25%       0.070000             7.000000             22.000000     0.995600   \n",
       "50%       0.079000            14.000000             38.000000     0.996750   \n",
       "75%       0.090000            21.000000             62.000000     0.997835   \n",
       "max       0.611000            72.000000            289.000000     1.003690   \n",
       "\n",
       "                pH    sulphates      alcohol      quality  \n",
       "count  1599.000000  1599.000000  1599.000000  1599.000000  \n",
       "mean      3.311113     0.658149    10.422983     5.636023  \n",
       "std       0.154386     0.169507     1.065668     0.807569  \n",
       "min       2.740000     0.330000     8.400000     3.000000  \n",
       "25%       3.210000     0.550000     9.500000     5.000000  \n",
       "50%       3.310000     0.620000    10.200000     6.000000  \n",
       "75%       3.400000     0.730000    11.100000     6.000000  \n",
       "max       4.010000     2.000000    14.900000     8.000000  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于数据的描述性分析就这么简单，我们通过Dataframe.describe()方法获得了一堆描述性统计结果。pd.read_csv()读取.csv文件。\n",
    "\n",
    "下面我们在用matplotlib看看，获得更直观的认识："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.font_manager import FontProperties\n",
    "font = FontProperties(fname=r\"c:\\windows\\fonts\\msyh.ttc\", size=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "设定绘图相关参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEVCAYAAADjHF5YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucHGWd7/HPb3omyRASxoEJmpghEHOCHkIS7IVo1AMq\nRhBljFzMGu+KsspxF41LhF284EF3FDmrvlwRERUMqwhzXC8E5LIKmhwHgoRVIgKBMBEYLpHbAMPk\nt39U9aS7p7q7uqere2bq+3695jXTTz9Vz6+qn/519VPPVJm7IyIiU19LswMQEZHGUMIXEUkJJXwR\nkZRQwhcRSQklfBGRlFDCFxFJCSX8KcTMppnZPgmuf18zOymhdS8wsz/XuOzzVdZ/j5ldWEtb9WZm\nGTObaWZdZnaQmR1mZm8wsxObHVuOmV1sZmvDvy83sxc0OyapTWuzA5C6ehnwbTP7DHA60A7MA3YB\nbcBfgFuBfwUWAvsBm4BZBB/+ZwLfAh4Cdrn7djN7D9ARrj8DrDezFwO7w7K73f0nxYGY2QuBvd19\nTBI3s0OB3wMr3f03ddjucTMzA15RLp5wX8xy969WWNd24P6Ip1rdfUVY533A+rDeM+HPUwSv1WPA\nY2bW6u5jPszM7JXAJnffXfxcXp3VwKeBFwDPA19z9y+XizsOdz8hr40zgW+6+8OVljOzhcBT7v7A\neGOQ2inhTyHufquZfQxY7O5HmtlLgI8CNwD7ufuFAGbWC/wB2Av4FHATQV/oApYSJJx7gO3AKcAn\ngVcBTwA9YXOnAt8AHi2Ow8xage8C7y8R6juB3wBrw99N5+5uZm82sznu3lei2gHAWjN7R4nnT3P3\n3wEPuPurip80s015D1uAL+a9JnsBF7j7R8vFaWbHU+GD0sxeDfwLcLy7/5eZ7Qf80MyedPdvllt/\nlT5I8DrHMQR818zeFPUhJg3i7vqZIj/AW4C9wr+3AZsJjiBvDx/fAqwiSN5fJ3izHg2cC/wyrPvr\n8O9jwvXcAFwC3AHcFj73S+CR8PfpEXGsBf6pRIwtwL3AEuBBoC0sXwD8Oa/emWHM9wJvCpf7fFh2\nN/CZvLrPE3wo3RVu49yw/ADgKuDPBN9sVobl7wEujIhtFvD/gUyJ2C8C/ibG67CpUjnwgfA1uDH8\n+Q3B0f2NeT97R+y7zQTfMsq1fw3w5qKyg4D7wr8vBtbm77/w90zgF+F+vB1YUlyf4CDgxcCVwDDB\ngcFpYeyzwjqLgNsi4joTeHez3ydp/tER/tQyE/iVmR0N3Al8BngzQRLcL6yTG+Z5ATAj/PtZggTz\nKMGb/VmCZAmAu681s48CIwQJAeA77v76EnGsAf53iedeC9zl7lvN7I/AMUDBkJCZrQHeALwceJpg\nSOndwGEEHxStwC/N7Hfu/lOCoaYH3H2hmX0PeB9wDvA9gsT+fTPLAj82s/9Raue5+xNmdgeQJUis\nxbqAi8zs2Yjnvut7hnr2MbMbI+o8XfT4Fnf/SLjNewN9ZfYpBPvjT+7+RJk6EHxL+8/8Ane/28ym\nVzjHY8CZ7n5L+Hp/nODDcQx3f2s4dPVqd7/fzF5DcDBxOfBWYEPEYhvYc6AhTaCEP4W4+wYz2+zu\njwVD0nwIuBCYDryXYKimz90vMrMVwHXufrGZ/TPwmnA1neHvvw9/W/j7EeB/AfuGjwfKhHKQu99V\n4rm1wI/Cv38UPi4+B3A88BV3fzJ8/KiZHQt8w92fA54zs+8SfHj8FHD2JJhfA1kzmwn8T3f/frhv\n+s1sJ7C4TNwQfBM4lIiE7+5vjlrAzE7IX6+7v7RCGwDTgCfM7JJw2WfDdd0IvBD4tLtfUrTMMmBL\njHW3EeyTYk7wbSiSuz9pZgeE5xf+Bng8Rls5GwiG+y4Pf48Z9go/dA6sYp1SZ0r4U8/RZnYtwZH7\nNe6+CcDMRoCTwr9fALySIDE+S5DgvknwFR1gNXtO1LaER3u5sfuX5Boys1Pc/YKIGCyiDDNrJzj6\nO9bMziYYotjbzGYXVZ3B2MTUyp4TxTkj4e/d7p6LfZjgiL9U3x4pUZ7zJLB3hTrF9iH4MMXMbicY\n3oDgvMnBZvYI8MdcfO5+VNjGX8KyE4GdwGfd/Qwz+3CJdmYRnEepZCuwkmA4izCuA4Gn3f2pcFZT\nJixvy6vzYYLX+R+Bqyn9LS3Kz4Evmdl8glMi91SxrDSIpmVOPacBDwPdwFvN7AYzu4FgiGNVWOef\ngO8AXyFIVpHCmTYPu/vXCMavc0Mw6wn6zjUlFt1tZlF9q4dgHHuOu7/Q3ecQnDA+oajetcCp4RBE\nq5nNCds6NZx62k5w4vcqSnD3vwJ3506wmtlhBN9e/lRqmdAcYDC/wMyOMbP+Ej8/IRguy528fsDd\nX+XBSdvcjJQteWW5D5wFBOPhObOp/O3jEYJhpUrOAc4PT9oT7r+LgC+Ez28n+LYAwbeknEOA6939\n9+zpK+UMAZ1mZu7+DPAr4IvAv0dVNrMMlT9wJUE6wp9CzOwQ4F5332Vmv3X344qe/0D451aCo8v9\n3P1rZnYkwQdF7gi6m+CI/xiCIRLc/QdmdjBBwhwGXu/u95UI5RaC8ebfFZW/k+AEcL6LCGZ7XJdX\n9g2CE39/IkgqpwIXhGV3AM8RzGi5tvTeAIJhhQvDaaqDwNvdfTgc7iplJXBpfoG7/4I95y7GMLMN\nBCe3w4c2I/d3mXaWEWxLH8E3gvcCB5vZeqCfYGpssZuA/1su+DDeq83sc8B/hLN/dhOc38kNnX0L\n6DOzn1M4S+o7BLN53gVsrNROWP8XwBnA94HLgP8gmBIcJUuwbdIk5q7r4U8V4Vfyp8KTlI8QJPZ8\nLwL+0d37zOyNBAn/EjO7nGAs/R3A2wiGDt5GkPi+SHDibj5BAr+E4I17AsG0zsvd/V+L4ng18B53\nLzUtc0IK54p/1d2PrVDvBQSJa4hgaOTPQI+7j4RDOrl56S8Mh3QGCf7vAIJvRmcQzCr6NMG5k+MJ\nZh99nOAk+z8AvwXOcfeCcXQz2wh8uNohk3BY7hPA33pC//tgZocD/6fUiWcz+w7wraTal8qU8KUk\nM5tFMJb+wqgEE35F3ytq1oiZfRO4xN1/nXyk4xcOQf0YOMPdt1WxjHuVbyIz25dgeOlggnMlV7n7\nYN7zLQQfqFfmnZvIPfdS4HPAidW2myQLvjZtIJgUcFnE80cSfMMqdX5CGkAJX0TGzcz+QnCe5d0T\n6YNICinhi4ikhGbpiIikxISapbPffvv5ggULmh2GiMikcfPNNz/s7nGm606shL9gwQL6+zVrS0Qk\nLjO7N25dDemIiKSEEr6ISEoo4YuIpIQSvohISijhi4ikhBK+iEhKJDot08z+geBWbk5wIa/3hpdR\nlTrr2zJA78Zt7Nw1xNyOdtatWkzP8nlV15mozurbyobNOxhxJ2PGmiPmc07PkorLFW/zUQd3cf0d\ng1Xvg6j2swd0Nn1/xtk+YEycxWVx90utfajRfa/W/jLVJXZpBTObR3DbvJe5+5CZ/RD4ubtfXGqZ\nbDbrmodfvb4tA6y/YitDw3suNd7eluHc1UtG31Rx6kxUZ/Vt5ZJNY6/EvHZFd9k3cdQ2F4uzD0q1\n32KwO+/t0+j9GWf72loMDIZH9gTaljFwGN5d+r0ftS219qFG971a+8tkZWY3u3s2Tt2kh3RagXYz\nayW4lO7OhNtLpd6N28a86YeGR+jduK2qOhPVhs07qirPidrmYnH2Qal2ivNlo/dnnO0b3u0FyR6C\n5F8u2UP0ttTahxrd92rtL2mQWMJ39wHgS8B9BDfb+Ku7X11cz8xOyd09aHBwsPhpiWHnrqGK5XHq\nTFQjJb6FlirPibttlepVaqeWNush6baK119rH2p036u1v6RBYgk/vEnE8cCBwFxgppmtLa7n7he4\ne9bds11dsS4HIUXmdrRXLI9TZ6LKlLhDVanynLjbVqlepXZqabMekm6reP219qFG971a+0saJDmk\n83rgHncfDG/icAXBjbOlztatWkx7W6agrL0tM3pyLm6diWrNEfOrKs+J2uZicfZBqXZaivJHo/dn\nnO1ra7FgzD6/LGPB2H4ZUdtSax9qdN+rtb+kQZKzdO4DVoT31BwCXofuZ5mI3ImvcrMg4tSZqHIn\n2qqddRG1zbXM0inVfrNn6cTdvuI6tc7SqbUPNbrv1dpf0iDRG6CEN48+meA2eVuAD7j7s6Xqa5aO\niEh1qpmlk+g8fHc/Gzg7yTZERCQe/aetiEhKKOGLiKSEEr6ISEoo4YuIpIQSvohISijhi4ikhBK+\niEhKKOGLiKSEEr6ISEoo4YuIpIQSvohISijhi4ikhBK+iEhKKOGLiKSEEr6ISEoo4YuIpIQSvohI\nSijhi4ikhBK+iEhKKOGLiKSEEr6ISEoo4YuIpIQSvohISijhi4ikhBK+iEhKKOGLiKSEEr6ISEoo\n4YuIpIQSvohISijhi4ikRGtSKzazxcC/5xUdBPyzu5+fVJsTyVl9W9mweQcj7mTMWHPEfM7pWVKX\n5aLqZA/opHfjNnbuGmJuRzvPj4zw4BPPjS6zaM5M7hl8iud9z3paDd5+RHfBumZOa+HxZ0cKlvvI\nUYsK1r1u1WKAgrKBXUMVt2329Ayf7VlScbn9Z00riL34cZRFc2Zy50NPjSmfkTGeGdmz0Qa0mBXs\nu59sGSjY5tnTMwAFZXFVitWAjDHmdTiwKzr+/O2Y1toyJs4nnh3Bi+pmirYPGNNfLv/djoL9Umzl\nwk4O7Nq7pj7ct2WgYn9Zt2oxPcvnFdQv7gsrF3Zy6QdfUbG9uDHk2kszcy/9otetEbMMMAAc4e73\nlqqXzWa9v78/8XiSdlbfVi7ZdN+Y8rUrusu+YeIsV6pOi8Hu5F9KANpaDAyGyyQMmZoq9eG+LQOs\nv2IrQ8N7Ppii+kt7W4ZzVwfrKa6fr5akHxVDrr2pmPTN7GZ3z8ap26ghndcBd5VL9lPJhs07qiqv\nZrlSdRqV7AGGd7uSfUpV6sO9G7eNSd5R/WVoeITejdsi6+e76a5Hq44xap259tIusSGdIm8HNkQ9\nYWanAKcAdHd3NyicZI2U+NZUqrya5SqtQyRJlfrfzhhDe7XUrUap9SbV3mSS+BG+mU0D3gL8KOp5\nd7/A3bPunu3q6ko6nIbImFVVXs1yldYhkqRK/W9uR3vsdc3taK+qfjXrraY8TRoxpHMMcIu7P9iA\ntiaE3ImyuOXVLFeqTksDPwfaWoy2jD540qhSH163ajHtbZmCsqj+0t6WYd2qxZH1861c2Fl1jFHr\nzLWXdo1I+GsoMZwzVZ3Ts4S1K7pHj4YyZhVPdsVdrlSd805axryOdgyY19HO/rOmFax70ZyZtBbl\n6FZjzLpyM1Tylzv/5MJ19564lN4TlhaUxTF7embMuqIUx178OMqiOTMjy2cUJZpglkzhvive5tnT\nM2PK4qoUq0Hk61Aq/pwZmbGvzezpGaI+dou3L6q/FO+XYisXdtbUh3uWz+Pc1Usq9pfcCdT8+lEx\n1DJLJyqGqXrCtlqJztIxs5nAfcBB7v7XSvWnyiwdEZFGqWaWTqInbd39KWDfJNsQEZF49J+2IiIp\noYQvIpISSvgiIimhhC8ikhJK+CIiKaGELyKSEkr4IiIpoYQvIpISSvgiIimhhC8ikhJK+CIiKaGE\nLyKSEkr4IiIpoYQvIpISSvgiIimhhC8ikhJK+CIiKaGELyKSEkr4IiIpoYQvIpISSvgiIimhhC8i\nkhJK+CIiKaGELyKSEkr4IiIpoYQvIpISSvgiIimhhC8ikhJK+CIiKaGELyKSEq1JrtzMOoALgUMA\nB97n7r+tZxt9Wwbo3biNnbuGmNvRzrpVi+lZPq/icmf1bWXD5h2MuJMxY80R8zmnZ0nF5Rac8bMx\nZTMyxjMjXvKxTD6tBs+XeQmNoEPXYtGcmdz50FMFj3c88nTZPrP/rGn89enhWP0sY1bQr3+yZYDH\nnx0ZfX729AxPPzdSdvtWLuzk0g++ouK2RL3/+u99dMx7K3tAZ0G9ow7u4vo7BguWA2p6L0c5+rwb\nxuzja04/sqZ1TSXmnlxiMrPvAr929wvNbBqwl7vvKlU/m816f39/7PX3bRlg/RVbGRre05nb2zKc\nu3pJ2Y5yVt9WLtl035jytSu6yyb9qGQvMlVVSvpR778Wg90RKaVUeU5bxsBhOK9SnPdylOJknzNV\nk76Z3ezu2Th1ExvSMbN9gNcA3wZw9+fKJfta9G7cVtDZAIaGR+jduK3schs276iqXCSNbrrr0bLP\nR73/SiX1cskeYHjEC5I9xHsvR4lK9uXK0yTJMfwDgUHgO2a2xcwuNLOZxZXM7BQz6zez/sHBwaoa\n2LlrqKrynJES32pKlYvIWJXeZ5OljTRJMuG3AocB33D35cBTwBnFldz9AnfPunu2q6urqgbmdrRX\nVZ6TMauqXETGqvQ+myxtpEmSCf9+4H533xw+vpzgA6Bu1q1aTHtbpqCsvS0zegKolDVHzK+qXCSN\nVi7sLPt81PuvpcQxU6nynLaM0VZUKc57OcqiOWMGEsqWp0liCd/dHwB2mFnuFXsd8Id6ttGzfB7n\nrl7CvI52DJjX0R7rJM85PUtYu6J79Ig+Y1bxhC3A9i+8KbJ8RsbKPpbJp7XCSzieV7g48SyaM7Ni\nn9l/1rTY/ay4X8+eXpiUZ0/PVNy+OLN0ot5/5520LPK9dd5JywrqrV3RXfC494Sl9J64tOr3cpRr\nTj8ych9PxRO21Up6ls4ygmmZ04C7gfe6+2Ol6lc7S0dEJO2qmaUTax6+mb0Z+Jm7764mEHe/FYgV\niIiIJCvukM7JwJ1m9i9mdnCSAYmISDJiJXx3XwssB+4CLjaz34bTKWclGp2IiNRN7JO27v44wUyb\ny4AXAW8FbjGz0xKKTURE6ihWwjez483sSuAGoA043N2PAZYCH08uPBERqZe4F09bDXzF3X+VX+ju\nT5vZ++sfloiI1FvcIZ0HipO9mX0RwN2vrXtUIiJSd3ET/tERZcfUMxAREUlW2SEdMzsV+DtgoZnd\nlvfULOCmJAMTEZH6qjSG/wPgF8C5FF747Al3L3/tVBERmVAqJXx39+1m9pHiJ8ysU0lfRGTyiHOE\nfxxwM8Ed3fIvueTAQQnFJSIidVY24bv7ceHvAxsTjoiIJKXSSduy169391vqG46IiCSl0pDOl8s8\n58Br6xiLiIgkqNKQzlGNCkRERJIV99IKmNkhwMuAGbkyd/9eEkGJiEj9xb0BytnAkQQJ/+cE/2V7\nI6CELyIyScS9tMIJBPekfcDd30twlcx9EotKRETqLm7CHwpvb/i8mc0GHgLmJxeWiIjUW9wx/H4z\n6wC+RfBPWE8Cv00sKhERqbtYCd/d/y7889/M7CpgtrvfVm4ZERGZWOKetH1NVFnxNfJFRGTiijuk\nsy7v7xnA4QRDO/rHKxGRSSLukM6b8x+b2Xzg/EQiEhGRRMSdpVPsfuCl9QxERESSFXcM/6sE186B\n4ENiOaALp4mITCJxx/DvADLh348AG9xdtzgUEZlEKl0euQ3oBd4FbA+L9we+CtxkZsvc/dZEIxQR\nkbqIc3nkvYAD3P0JgPA/bb9kZt8A3gjo5igiIpNApYR/LLDI3XPj97j742Z2KvAwwUXURERkEqg0\nS2d3frLPcfcRYNDdNyUTloiI1FulI/w/mNm7iq97b2ZrgT9WWrmZbQeeAEaA5909W2ug1ejbMkDv\nxm3s3DXE3I521q1aTM/yeRWXO6tvKxs272DEnYwZa46Yzzk9SwrqLDjjZ0mFPeW0GjzvhY9bW4xn\nRsYcQ4wy9kwHq8aMjPH8bh/T3r57T+PBJ54r294rF3Zy012PjpatXNjJidnugj501MFdXH/HYEGf\n6r/30TH9JXtAJ70btzGwa4iM2ehzI+7MC5cDxvTPqLI4fbZY3L5fXG/Bvu1suvuxgm3ZfPcj3PnQ\nU6PLLJozk2tOP7LqmKS0WnNVrSziAH7Pk2bzgCuAIYL/rAXIAu3AW919oOzKg4SfdfeH4wSTzWa9\nv78/TtWS+rYMsP6KrQwNj4yWtbdlOHf1krI78qy+rVyy6b4x5WtXdI8mfSX79Ggx2F3mkyfTYoxE\nVChVntPWYmAwnPeh15YxcBjOWy5Ony0Wt+9H1YtLSb9+as1Vxczs5rgH02WHdNx9wN2PAD5LMEtn\nO/BZdz+8UrJvlt6N28Z05KHhEXo3biu73IbNO6oql6mtXLIHSib1cskegqQ+XPQNZ3jEC5I9xOuz\nxeL2/ah6ceUf8cv41JqrxiPupRWuA66rYf0O/NLMRoBvuvsFxRXM7BTgFIDu7u4amii0c9dQVeU5\nIyW+6ZQqF0lapT4bt35xebXrlWTUmqvGo9ZLK8T1KndfRjCb5yNRV9109wvcPevu2a6urnE3OLej\nvarynIxZVeUiSavUZ+PWLy6vdr2SjFpz1XgkmvBzwz7u/hBwJcFVNhO1btVi2tsyBWXtbZnRE2Ol\nrDki+gZepcplamup8DmfKVGhVHlOW4sFY/b5ZRkLxvbzxOmzxeL2/ah6cS2aM7Om5WSsWnPVeCSW\n8M1sppnNyv0NvAG4Pan2cnqWz+Pc1UuY19GOAfM62mOdBDmnZwlrV3SPHtFnzApO2AJs/8Kbkgx9\nymm1sY9nZMonxFq/T83IWGR7+8+aVrG9lQs7C8pWLuzkvJOWFfShtSu6Cx5/+cSlkf3lyycuZV54\nhJb/HOFyvScupfeEpQXr6j1hKb0nLq26zxaL2/ej6q1c2DlmW4qTu07Y1letuWo8ys7SGdeKzQ4i\nOKqH4FzBD9z98+WWqccsHRGRNKlmlk7ci6dVzd3vBpYmtX4REalO0idtRURkglDCFxFJCSV8EZGU\nUMIXEUkJJXwRkZRQwhcRSQklfBGRlFDCFxFJCSV8EZGUUMIXEUkJJXwRkZRQwhcRSQklfBGRlFDC\nFxFJCSV8EZGUUMIXEUkJJXwRkZRQwhcRSQklfBGRlFDCFxFJCSV8EZGUUMIXEUkJJXwRkZRQwhcR\nSQklfBGRlFDCFxFJCSV8EZGUUMIXEUkJJXwRkZRQwhcRSYnWpBswswzQDwy4+3FJt5e0s/q2smHz\nDkbcMWCvaRmeem5k9PmMGWuOmM89g09y012PjpbvP2saDz85zIj7aB1gdF25snN6lhS0d+jZV/H4\ns3vWP3t6hrcsnzdmueL2Fs2ZydPP7WbnriHmdrSzbtVivn79ndz50FMFdT5y1CJ6N24rqNezfF5B\nDH1bBsbU+VH/fRXbK15PXO/41m8L1r1yYScPPfFsQez7z5pGayZTNqaVCzs5MdtdEPuCfdvZdPdj\nVb8OUaL2S5x9V+t+mQim2vakjbl7sg2YnQ5kgdmVEn42m/X+/v5E4xmPs/q2csmm+xJtY+2K7tFk\nU5zsk2BAfg9ob8tw7uolo2/ivi0DrL9iK0PDe+JoMdhdodsUryeu4mQfV5yYqpH/OkSJ2i9x9l2t\n+2UimGrbM1WY2c3uno1TN9EhHTN7MfAm4MIk22mUDZt3NLSNpJM9FCZ7gKHhEXo3bht93LtxW8Eb\nHOIl1uL1xFVLsof6Jnuo/FpH7Zc4+67W/TIRTLXtSaOkx/DPBz4J7C5VwcxOMbN+M+sfHBxMOJzx\nGUn421Cj2qhk566hyL/Hs57JptLrUGrb4uy7ybpfptr2pFFiCd/MjgMecveby9Vz9wvcPevu2a6u\nrqTCqYuM2ZRoo5K5He2Rf49nPZNNpdeh1LbF2XeTdb9Mte1JoySP8FcCbzGz7cBlwGvN7JIE20tc\n7gRfo9qYPT2TeHvFaa29LcO6VYtHH69btZj2tsI4WmJ8JhWvJ66VCzurXgbixVSNSq911H6Js+9q\n3S8TwVTbnjRKLOG7+3p3f7G7LwDeDlzn7muTaq8RzulZwtoV3aNHfwbMnFb4BsiYsXZF95jEtf+s\naaPL5erkrytXln+i8LbPvHFM0p89PRO5XHF7i+bMZF5HOwbM62jn/JOXsWjOzDF1vnLysoJ6xSfg\nepbP49zVSwrqnHfSsort1Xoi79IPvmLMulcu7BwT+/6zplWMaeXCTs4v2r6VCzurfh2iRO2XOPtu\nMp/gnGrbk0aJz9IBMLMjgU9M9lk6IiITTTWzdBKfhw/g7jcANzSiLRERiab/tBURSQklfBGRlFDC\nFxFJCSV8EZGUUMIXEUkJJXwRkZRQwhcRSQklfBGRlFDCFxFJCSV8EZGUUMIXEUkJJXwRkZRQwhcR\nSQklfBGRlFDCFxFJCSV8EZGUUMIXEUkJJXwRkZRQwhcRSQklfBGRlFDCFxFJCSV8EZGUUMIXEUkJ\nJXwRkZRQwhcRSQklfBGRlFDCFxFJCSV8EZGUUMIXEUkJJXwRkZRoTWrFZjYD+BUwPWzncnc/u97t\n9G0ZoHfjNnbuGmJuRzvrVi2mZ/m8ejfTtPbixgAUlB11cBfX3zHIzl1D7NPehhnsenq4YsxJb1+c\n9cfZvmbsd5HJztw9mRWbGTDT3Z80szbgRuBj7r6p1DLZbNb7+/tjt9G3ZYD1V2xlaHhktKy9LcO5\nq5ckkgwa3V7cGNoyBg7Du+O9lqViTnr74qw/7vY1er+LTFRmdrO7Z+PUTWxIxwNPhg/bwp+6frr0\nbtxWkBgAhoZH6N24rZ7NNK29uDEMj3jsZA+lY056++KsP+72NXq/i0wFiY7hm1nGzG4FHgKucffN\nEXVOMbN+M+sfHBysav07dw1VVT5ejW4vybai1pP09sVZfzVtNXK/i0wFiSZ8dx9x92XAi4HDzeyQ\niDoXuHvW3bNdXV1VrX9uR3tV5ePV6PaSbCtqPUlvX5z1V9NWI/e7yFTQkFk67r4LuB54Yz3Xu27V\nYtrbMgVl7W2Z0ZN89dbo9uLG0JYx2los9jpKxZz09sVZf9zta/R+F5kKkpyl0wUMu/suM2sHjga+\nWM82cifsGjV7o9HtVRNDcVkts3SS3r4464+7fZqlI1K9JGfpHAp8F8gQfJP4obt/ttwy1c7SERFJ\nu2pm6SR2hO/utwHLk1q/iIhUR/9pKyKSEkr4IiIpoYQvIpISSvgiIimhhC8ikhKJTcushZkNAvc2\nO46Y9gP0I+7yAAAFRklEQVQebnYQNVLsjTdZ4wbF3gzVxH2Au8e6TMGESviTiZn1x537OtEo9sab\nrHGDYm+GpOLWkI6ISEoo4YuIpIQSfu0uaHYA46DYG2+yxg2KvRkSiVtj+CIiKaEjfBGRlFDCFxFJ\nCSX8GpjZx8zsdjP7LzP7+2bHU46ZXWRmD5nZ7XllnWZ2jZndGf5+QTNjjFIi7hPDfb7bzCbsVLsS\nsfea2R1mdpuZXWlmHc2MsZQSsX8ujPtWM7vazOY2M8YoUXHnPfdxM3Mz268ZsVVSYp9/2swGwn1+\nq5kdW4+2lPCrFN6m8YPA4cBS4Dgze0lzoyrrYsbeaewM4Fp3XwRcGz6eaC5mbNy3A6uBXzU8mupc\nzNjYrwEOcfdDgT8B6xsdVEwXMzb2Xnc/NLxd6U+Bf254VJVdTMQd9cxsPvAG4L5GB1SFi4m+G+BX\n3H1Z+PPzejSkhF+9lwKb3f1pd38e+E+CJDQhufuvgEeLio8nuDkN4e+ehgYVQ1Tc7v5Hd9/WpJBi\nKxH71WF/AdhEcJ/nCadE7I/nPZwJTLiZHiX6OcBXgE8yAWPOKRN73SnhV+924NVmtq+Z7QUcC8xv\nckzV2t/d/xL+/QCwfzODSaH3Ab9odhDVMLPPm9kO4B1MzCP8MczseGDA3X/f7FhqdFo4lHZRvYZd\nlfCr5O5/JLg379XAVcCtwEhTgxoHD+blTtijn6nGzM4EngcubXYs1XD3M919PkHcH212PJWEB2Of\nYpJ8OEX4BnAQsAz4C/DleqxUCb8G7v5td3+5u78GeIxgTHYyedDMXgQQ/n6oyfGkgpm9BzgOeIdP\n3n+AuRR4W7ODiGEhcCDwezPbTjCEdouZvbCpUcXk7g+6+4i77wa+RXDOcNyU8GtgZnPC390E4/c/\naG5EVfsJ8O7w73cD/6+JsaSCmb2RYCz5Le7+dLPjqYaZLcp7eDxwR7Niicvdt7r7HHdf4O4LgPuB\nw9z9gSaHFkvugCz0VoKh5PGvd/IeaDSPmf0a2BcYBk5392ubHFJJZrYBOJLgcqsPAmcDfcAPgW6C\ny1Gf5O4NOWkUV4m4HwW+CnQBu4Bb3X1Vs2IspUTs64HpwCNhtU3u/uGmBFhGidiPBRYDuwn6y4fd\nfaBZMUaJitvdv533/HYg6+4T7lLJJfb5kQTDOQ5sBz6Ud96t9raU8EVE0kFDOiIiKaGELyKSEkr4\nIiIpoYQvIpISSvgiIimhhC+pYGY94RUTDw4fL4i6smLMdW2v5sqLZvYeM/taLW2J1JMSvqTFGuDG\n8LdIKinhy5RnZnsDrwLeD7w94vmMmX0pvMfBbWZ2Wlj+OjPbYmZbwwtYTc9b7DQzuyV8LvetodPM\n+sJ1bDKzQxuxfSJxKeFLGhwPXOXufwIeMbOXFz1/CrAAWBZer/5SM5tBcJ3yk919CdAKnJq3zMPu\nfhjBRa4+EZZ9BtgSruNTwPcS2h6RmijhSxqsAS4L/76MscM6rwe+mbtefXiZicXAPeGHBAT3DXhN\n3jJXhL9vJviwgOBbxPfDdVwH7Gtms+u3GSLj09rsAESSZGadwGuBJWbmQIbg+iRfH+eqnw1/j6D3\nkUwSOsKXqe4E4PvufkB45cT5wD0U3rTmGuBDZtYKox8S24AFebevfCfB3c3K+TXBDUIwsyMJhn0e\nL7uESAMp4ctUtwa4sqjsxxTeU/ZCgnue3mZmvwf+1t2fAd4L/MjMthJcKfLfKrT1aeDlZnYb8AX2\nXIJaZELQ1TJFRFJCR/giIimhhC8ikhJK+CIiKaGELyKSEkr4IiIpoYQvIpISSvgiIinx32707mgM\nPn5kAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x288fb65e630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(df['alcohol'], df['quality'])\n",
    "plt.xlabel('Alcohol')\n",
    "plt.ylabel('Quality')\n",
    "plt.title('酒精度（Alcohol）与品质（ Quality）',fontproperties=font)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "散点图显示酒精度（Alcohol）与品质有比较弱的正相关特性，整体呈左下-右上趋势，也就是说酒精度较高的酒具有较高的品质。挥发性酸度（volatile acidity）与品质呈现负相关特性："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEVCAYAAADjHF5YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYHGWZ9/HvbyaNdACZABHNmBBh3eAhhOgIUcAFDxtB\n0YCwEI2sqLC6+vrKutkVZRVdVnRnVTzsqsgruovGA4d4QuMJBNREJwQIKHEBkTBRiUCUw6jD5H7/\nqJqh09OH6klXz0zq97muuWa66qnnubsOd1c99Uy1IgIzM9v1dU12AGZm1hlO+GZmBeGEb2ZWEE74\nZmYF4YRvZlYQTvhmZgXhhD8FSZohaX7697htJKlLkiZY9xGSntNg/r6S/mYidU8wnnMlndNC+RdL\nOiv9+zOSVuQU01E1ps+XdFt1HNOBpG5Je0iaLelASc+Q9NeSTp7s2EZVbk9Jl0qaNdkx7WpmTHYA\nRSDpPODrwBLgdcC2itl7AVdExLmS/gc4A3gq8A/ACuAsSfsD/xIRf0qXeSdwNXC1pCXALcA3I+LI\nijY/ADwf2Bt4CLgBOBM4B3idpDdGxH+mZV8N9KSLdgNnS3oisD2ddkdEfLXG+3o8sGdE3DbRdZOF\npHcAn4yI30XEN4Bv5NleRJybocxYHJIWAs+KiE83Wy79oH52RPyoQZlXA3tFxEeb1HUncHeNWTMi\nYkla5jXA2Wm5P6Y/D5Hsg/cD90uaERGP1Kj/OcDaiNhePa+izInAucAs4BHgYxHxgUZxZxERJ1W0\nMbb9my0n6SDgoYj4zc7GsCvyGX7OJK0EXkxy0O1BktBvAf42TdCvBLZIWgTsFhF/BBaRJGjSg+eX\nwLOqqj48/f0RYKS63Yh4K0ly/xzwPuA04NPAF4EDgNMkHZcWPxMYAHYnSQjLgJ+kbQ4A4xK6pBnA\nZ9PyeTsjjW2qeiZQ96qpUiT/6Xi8pGUNih0AvFnS2jo/o/vCbyLiyOqfqrq6gPdHxDERcSzJ/kZE\nvCki/iUiPlgn2b8MWNYk2R8F/DuwPCLmkuwvL5b0d1nWRQta2f5DwGfT/dOqeKXk70Lg8cC3gacB\n/cBi4GkVvTJXAk8EDpC0FvhL4FeSTqqsSNL1wC+A+cBTJG0FnpDWvVDSdWnRT5N8CKwkuYJ4GNgf\n2JzGsFda5ulp238GXg/0pX/fk9azOF3uSuBnVe/rVOC6iNjhDFPSD4F3RsT3JJWAXwEHpzF/PF0X\nvwVeGxE/r1r2VJKzxTJweUScJekKoBe4VtK706JHRsTrqpZdnNa/L/AD4MzKZCVpD+DSdN0OkSSp\njZIeC3wUOAIQSeJ+P/DdiLgkTa6fAnYDvlNR36uBI0k+UN8P7C7pGem66oqIt6flPpe+l8sqwn0v\n8D1JX4uIcR/WJAn/FRHx0xrzJmJxxb7RBTy14jXAiyLiwYr31gW8HXhBk3rfCZwVEbcARMTvJL2O\n5Orzk5I+Q7oe03ofiYgZ9bZFZcXp1cuRJNtmdPt/EPhXYG5EPCDpycBlEXHI6HIRsUXSNSQfbJ/N\ntnqKwwk/ZxHxe0lDwO8j4oOSngC8G3heRJxdVfwb6SX/DSRdBLXOvD4H3AF8FXgDcEpE/EjSdVVd\nOm8DVqUvf0/STbQHSTL8DkkS66+Ic4WkN5F8UHwznXxxRNQ76JcDb64x/YvAy4DvkXQpXUfygXMp\ncEZE/CA9e/wfkg+YSr8kuboRcJukD0fECenBf1RE3J0m2up1MgP4BHBSRGyWtCqN4YrKYsA7IuL6\n9H2+FXh1ug62Ak9O188jVfV+AXhVuo7fVd12RFwl6Z9JP4QkPS1d5u3pB96RwGurlnlA0q3p+19X\nYx3OBj4t6U815n22oqtn76rEPerhqtfXR8Qb0/e0J7C6wXaF5IrlFxHxQIMykGyrH1ROiIg7JD1G\n0t4Nlqu3Lcapsf2fCywl2Z9O4NF9vNIq4D9xwh/HCT9nkvYCDgQulPQ+4LCIWJkmiVqOBhYAGyTd\nm07bl6Qv9QzgWpKz5PuA9zbqCyY5m78EeA2wH/Ax4G0k3TBfIrmqgOQABLgX+Ku0PYDBBnUfGBG3\n15j+ZeBHJB8GLyc5A14APBgRPwCIiK9I+kR6dl1pkOTgXwjsSXJmd2eDGEYdTHK18p30qqlM0iU1\nJiIelHRA2qf9LOAP6ayXAU9Ju1oeBKi48lpA8kE9uo4/B7yqUSARcYukkHQgyYfIVWk3XbUbgEOo\nkfAj4vhadadXfAsqyj2lUSyp3YAHJF2SLvuntK7rSPajc0fPwCscCmzIUHcJqPUwrqDig3PczPrb\nIotVJF2Ol6a/X1mj/jskPamFOgvDffj5+0+SZPQRkhtn3enB1ifpuvSnckTIW0l25LURcXREHE3S\nnfIpgIj4REXZ0mgdpF06VfUNRsSVJF0ONwCHpXX9AXgTSXIG6ErPtF4L/AXJh87RwOMlnVnnfdUc\nJRQRvwbuktQHHEVytTCDR28AjxWtnJaeTf+I5OrlrST3ObKORJoB3BoRB6c/B0TEh3YIVno98Hck\n6/HfKureHRiuU2/1vMdkjGcV8FLgRGqfgULy4bJnxvpG7U1yoxVJN1ds71vTafdWTLsqXWbPtC2A\nk4HnkXTFHQn8R5129gKand0DbCTpChuTJtqHI+IhkqTfnU4vVZSpty2yuBI4UtJcklsiv2xh2cJz\nws9ZRJxG0v1yc0RcU3FjbaDiRtu1AJJeBGyNiG8B90g6U9Jbgbsi4ic16v52RX0bq+sD/krS1cCH\nSM7u/kzSZdMFfD0iHlYy0uZ3EfExkn79nwPHktxk7qKi37rKdtUYMpr6Iklf/NUR8ee0zv1GP4gk\nHQ/cXtlvTDJKaE9gNUmiPaRi3hCwj1R3KOomoFfSs9P6F9foUng6ydn2jSRdAqO+B7wlXW6WpHLF\nvFuBA5XcUIckSdVSHd8XgOOBZ6f11/I4kq6kMZKOlTRQ5+erJFdp96XFf1Ox7UdHpGyomDZ6b2A+\nO14lPZaKq4Q67iXpVmrmPOACSX+Rxv84kn3ofen8O0muFiD5oBlVb1vUM7Z+06ula0hOYr5Yq7Ck\nbmoMZDB36XSKACR9mUcPpEVpMoYkYb2HpG//hHTaR4HRm3ZLJtjuaRFxq6Qekkv3cySdTpKQ/jIt\ncyxJNxER8XlJB5PcGB4GXhARd9Wp+3qSvt5aNxYvBT5MepBHxJ+VjO3/rzQR30kyamhMesPvyyQj\ngn4C3Fwx+2KSK4W31QokIoYknQZ8Jr1SuIPk7LrSxcCX0nJrKqa/Cbgo7Se+n4oblRHxUHqF85X0\nPszFtVcFV5Fsu3UkXXa/TD84rq11HyZ1BEkXUeX7+CaP3j8ZJ703cfWjLzU6cqXRGfKhJB9cq0mG\nYp4OHCzpbJIRWPfUWOaHJNuvoYj4tqR/Bb4maSbJFdseJCc4kJzBr5Z0JcnV26h626Keyu3/PyT7\n79dIhi7X0kfy3qyKws/Dz5WkD5OcxTw3Iu6pmP7dyhtnSsbgf5LkjOwEkjOx95EczO8hOYv7bNr/\nfS7JgX9eRVN/waPDJ/+X5EPkRJIbeDNIRsvcTdKdcxnJ2er1JIn3/SQ3zeaSJPBLSA6ak4CZwKUR\n8ZGq93UU8OqI2OGGpCXSJPfeiBh3U1XJWPGPRsRx45fcodwsksQ1RNI1chvJUMkRSTcDo+PSHx8R\nBysZtXVjOq2LJEH+E8nV1nNJ7ldsIukyOx44C/gxcF5E7NCPLmkN8PpWu0zSrsF/JBll1Oj+0oRJ\nOoxk3da88SzpYuBTebU/nTnh50xSd52hd9XlZpBchr6BpCvkZ1XzjwLurZ7eoD5FxcZNu19Gx/mP\nlSHpRnmEJGmMO7jTy+OZtUZsSPokcElFF5Ix9o9YnwcOiaoDLN0OlwFvi4hNGevrIumvbulglbQv\nsA/JTe0e4FsRsbVifhfJh/oVETFctexTSIZAntxqu3lK99lVJCONvlBj/tHAqRHx+k7HNh044Zu1\nkaS3kNwTyO0Mt8gk/ZrkvtLfTqUPounCCd/MrCA8SsfMrCCm1Cid/fbbL+bPnz/ZYZiZTRvr16//\nXURkGUY7tRL+/PnzGRjwaCozs6wk/SprWXfpmJkVhBO+mVlBOOGbmRWEE76ZWUE44ZuZFYQTvplZ\nQeQ6LFPSWSRf2h0kz84+vc6XQUw7qzcM0r9mE1u2DTGnp8zKpQtYtri3ZpnBbUN0S4xEjPvd21Nm\n5m5d/O89D+2wbG9PmWMOns1Vt24da6P69fx9y6y94/6x+pYfPpfzli1sW/yNymVdvpW2pqLpHLtZ\ntdwerSCpl+Tr7Z6aPr72S8CVEfGZesv09fXFdBiHv3rDIGdfvpGh4UefiVYudXP+iQvHkkGtMp2w\nYsm8pkk/S/yNyr38mb1ctn6w6fKttDUVTefYrTgkrY+I6q8LrSnvLp0ZQDl9EuRMYEvO7XVE/5pN\n4xL50PAI/Ws2NSzTCavWbW5aJkv8jcqtWrc50/KttDUVTefYzWrJLeFHxCDJV6jdBfya5LtBv11d\nLv1WpwFJA1u3bq2ePSVt2TbUdHq9MnkbyXDFliX+RuXqtVGrfNa2pqLpHLtZLbkl/PTLG14GPAmY\nA+whaUV1uYi4MCL6IqJv9uxMj4OYdHN6yk2n1yuTt+663wL4qCzxNypXr41a5bO2NRVN59jNasmz\nS+cFwC8jYmv65QqXA8/Jsb2OWbl0AeVS9w7TyqVuVi5d0LBMJyw/fG7TMlnib1Ru+eFzMy3fSltT\n0XSO3ayWPEfp3AUsSb/rcgh4PrvI90yO3rBrNHqjssxUG6WTJf5m5foO2CfT6JWsbU1F0zl2s1py\n/QIUSe8GTiH5Cr0NwOsi4k/1yk+XUTpmZlNFK6N0ch2HHxHvAt6VZxtmZpaN/9PWzKwgnPDNzArC\nCd/MrCCc8M3MCsIJ38ysIJzwzcwKwgnfzKwgnPDNzArCCd/MrCCc8M3MCsIJ38ysIJzwzcwKwgnf\nzKwgnPDNzArCCd/MrCCc8M3MCsIJ38ysIJzwzcwKwgnfzKwgnPDNzArCCd/MrCCc8M3MCsIJ38ys\nIJzwzcwKwgnfzKwgnPDNzArCCd/MrCCc8M3MCsIJ38ysIJzwzcwKYkZeFUtaAHyxYtKBwDsj4oI8\n2nvlp37MD2+/b+z1EQftw+fOeHbd8qs3DNK/ZhNbtg0xp6fMMQfP5qpbtzK4bWhc2d6eMg8M/Zk/\n/Gkkj9Bztf9eu7HuHS/knNUbWbVuMyMRE66rW2pp+d6eMvP3LbP2jvsZiaBb4sDZM7l960Nsb1DN\n/nvtxtnHPZX+NZsY3DY01m5vT5mVSxewbHHvWNnq7Vg9P4ta+8Jl6+9maHg7AF2CVxw+j/OWLZxQ\nfROJySwPip1IAJkbkbqBQeDwiPhVvXJ9fX0xMDDQcv3VyX5UvaS/esMgZ1++kaHh6ZfAJ2L3bvHH\nkfy3cyeUS92cf+JCli3urbkdK+dn0cq+sGJJ86TfjpjMWiFpfUT0ZSnbqS6d5wO3N0r2O6NWsm80\nvX/NpsIke2CXSfYAQ8Mj9K/ZBNTejpXzs2hlX1i1bvOE6ms1JrO8dCrhnwqsqjVD0pmSBiQNbN26\ntSPBbKnRbWPTx+j2q7cdW9m+rZTN0p3VjpjM8pJ7wpe0G/BS4Mu15kfEhRHRFxF9s2fPzjscAOb0\nlDvSjuVjdPvV246tbN9WynZLE67P+5xNBZ04wz8WuD4ifptXA0cctE9L01cuXUC51J1XOFPO7t3N\nE9V0US51s3LpAqD2dqycn0Ur+8Lyw+dOqL5WYzLLSycS/nLqdOe0y+fOePa45N5olM6yxb2cf+JC\nenvKiGQ0yYol8+itcxbW21PmsY+Znh8Q+++1G7f+23GsWDIv0xlqI60u39tT5oiD9hlbrlviyY/b\ng64m1ey/125ccMqhY9tjdPnenvIONz9rbcdWb47W2xfKpUcPjS5lu2HbrpjM8pLrKB1JewB3AQdG\nxO+blZ/oKB0zs6JqZZRObuPwASLiIWDfPNswM7Ns/J+2ZmYF4YRvZlYQTvhmZgXhhG9mVhBO+GZm\nBeGEb2ZWEE74ZmYF4YRvZlYQTvhmZgXhhG9mVhBO+GZmBeGEb2ZWEE74ZmYF4YRvZlYQTvhmZgXh\nhG9mVhBO+GZmBeGEb2ZWEE74ZmYF4YRvZlYQTvhmZgXhhG9mVhBO+GZmBeGEb2ZWEE74ZmYF4YRv\nZlYQTvhmZgXhhG9mVhBO+GZmBeGEb2ZWEDPyrFxSD3AR8HQggNdExI/zbLOec1ZvZNW6zYxE0C2x\nPYKoKtMtMRLVU21X1yXYHjBrZok/Do8wNLy96TI95RLnvvRpLFvcy+oNg7z7a7dw/8PDAIhkZ+/t\nKbNy6QKWLe6tWcfqDYP0r9nElm1DzOkpc8zBs7nq1q1jr+fvW2btHffvsE+O7qO9Nco3assMQJFj\ngpP0WeDaiLhI0m7AzIjYVq98X19fDAwMtD2Oc1Zv5JK1d7W9Xiu2Upc45bC5fPGnmxkeqX0clUvd\nnH/iwnGJePWGQc6+fCNDwyNti6deW7Zrk7Q+IvqylM2tS0fS3sBzgf8HEBF/bpTs87Rq3ebJaNZ2\nccPbg1Xr6id7gKHhEfrXbBo3vX/NprYm+0ZtmY3Ksw//ScBW4GJJGyRdJGmP6kKSzpQ0IGlg69at\nuQTibhrLS5Z9a8u2oUzT2iGvem3XkGfCnwE8A/h4RCwGHgLeVl0oIi6MiL6I6Js9e3YugXRLudRr\nlmXfmtNTzjStHfKq13YNeSb8u4G7I2Jd+vpSkg+Ajlt++NzJaNZ2caUusfzwuZS66yf9cqmblUsX\njJu+cukCyqXutsZTry2zUbkl/Ij4DbBZ0uge+HzgZ3m118h5yxayYsm8sbOxbolah6ivBIqpK93s\ns2aWKJeyHRI95RL9Jy/ivGUL6T9pEbNmlsbmje5FvT3lujdRly3u5fwTF9LbU0Zp2RVL5u3w+oiD\n9hm3T46+rlXeN2ytmbxH6RxKMixzN+AO4PSIuL9e+bxG6ZiZ7apaGaWTaRy+pOOBb0RE8wHKFSLi\nBiBTIGZmlq+sXTqnAP8r6d8lHZxnQGZmlo9MCT8iVgCLgduBz0j6cTqccq9cozMzs7bJfNM2Iv5A\nMtLmC8ATgBOA6yX9n5xiMzOzNsqU8CW9TNIVwNVACTgsIo4FFgFvzS88MzNrl6wPTzsR+FBEXFM5\nMSIelvTa9odlZmbtlrVL5zfVyV7S+wEi4nttj8rMzNoua8J/YY1px7YzEDMzy1fDLh1JbwD+HjhI\n0k0Vs/YCfphnYGZm1l7N+vA/D3wTOJ8dH3z2QETcl1tUZmbWds0SfkTEnZLeWD1D0j5O+mZm00eW\nM/yXAOtJvrWt8klOARyYU1xmZtZmDRN+RLwk/f2kzoRjZmZ5aXbTtuHz6yPi+vaGY2ZmeWnWpfOB\nBvMCeF4bYzEzsxw169I5plOBmJlZvrI+WgFJTweeCuw+Oi0i/juPoMzMrP2yfgHKu4CjSRL+lST/\nZXsd4IRvZjZNZH20wkkk30n7m4g4neQpmXvnFpWZmbVd1oQ/lH694SOSHgvcA8zNLywzM2u3rH34\nA5J6gE+R/BPWg8CPc4vKzMzaLlPCj4i/T//8hKRvAY+NiJsaLWNmZlNL1pu2z601rfoZ+WZmNnVl\n7dJZWfH37sBhJF07/scrM7NpImuXzvGVryXNBS7IJSIzM8tF1lE61e4GntLOQMzMLF9Z+/A/SvLs\nHEg+JBYDfnCamdk0krUP/1agO/37XmBVRPgrDs3MppFmj0cuAf3AacCd6eT9gY8CP5R0aETckGuE\nZmbWFlkejzwTOCAiHgBI/9P2PyR9HHgR4C9HMTObBpol/OOAJ0fEaP89EfEHSW8AfkfyEDUzM5sG\nmo3S2V6Z7EdFxAiwNSLW5hOWmZm1W7Mz/J9JOq36ufeSVgA/b1a5pDuBB4AR4JGI6JtooO12zuqN\nrFq3mZGKz7NuiZGIsd+9PWWOOXg2V926lcFtQzuUO3D2TO7Y+jAjEUhQntHFw8Pba7a1e7f448i4\nz02bZkb3i1aVumCPx5T4/dAwc+rsU5VlH9nOWLmv3/hrtg0NAzBrZokXH/KEsWW7BNvTcHrKJc59\n6dNYtriX1RsG6V+ziS3bhpjTU2bl0gUsW9y7U+99VKt1tzOWymO2W2L54XM5b9nCib6VQlKNE/hH\nZ0q9wOXAEMl/1gL0AWXghIgYbFh5kvD7IuJ3WYLp6+uLgYGBLEV3yjmrN3LJ2rtyb8esk0pd4pTD\n5nLZ+kGGhkfGppdL3Zx/4sKdTvqrNwxy9uUbM9fdavlG6h2zK5bMK3zSl7Q+68l0wy6diBiMiMOB\n95CM0rkTeE9EHNYs2U9lq9ZtnuwQzNpueHuwat3mHRIswNDwCP1rNu10/f1rNrVUd6vlG6l3zPpY\nbk3WRyt8H/j+BOoP4LuSRoBPRsSF1QUknQmcCTBv3rwJNNG6iVyWm00H9fbtLTW6j1pVr452TW+k\n3vvysdyaiT5aIasjI+JQktE8b6z11M2IuDAi+iKib/bs2TmHk+iWOtKOWafV27fn9JR3uu56dbRr\neiP13peP5dbkmvBHu30i4h7gCpKnbE665Yf7y7ps11PqSm5klkvdO0wvl7pZuXTBTte/cumClupu\ntXwj9Y5ZH8utyS3hS9pD0l6jfwN/DdycV3utOG/ZQlYsmTfu7GD09ejv3p4yK5bMo7fqjKRb4smP\n22OsnAQzS/VX5e7dPgvZFUz0bLLUlYyiEfX3qcqyleV6yqWxebNmlnZYtqsinJ5yif6TF3HesoWc\nf+JCenvKY/W044YtwLLFvS3V3Wr5RqqP2W7JN2wnoOEonZ2qWDqQ5KweknsFn4+If2u0TKdG6ZiZ\n7SpaGaWT9eFpLYuIO4BFedVvZmatyfumrZmZTRFO+GZmBeGEb2ZWEE74ZmYF4YRvZlYQTvhmZgXh\nhG9mVhBO+GZmBeGEb2ZWEE74ZmYF4YRvZlYQTvhmZgXhhG9mVhBO+GZmBeGEb2ZWEE74ZmYF4YRv\nZlYQTvhmZgXhhG9mVhBO+GZmBeGEb2ZWEE74ZmYF4YRvZlYQTvhmZgXhhG9mVhBO+GZmBeGEb2ZW\nEE74ZmYF4YRvZlYQTvhmZgUxI+8GJHUDA8BgRLwk7/babfWGQfrXbGLLtiHm9JRZuXQByxb3trXO\nYw6ezVW3bh3Xxmi5wW1DdEuMRIz97s0YS2Ud1XprtD1/3zJr77ifkYiG5UbbPmf1Rlat27xD+W6J\n5YfPBRg3r1JPucS5L33a2Hs4Z/VGPr/uLrZXFZ9Z6uK9Jx4y7r3W2jbAuHU2Gv/Xb/w124aGAZg1\ns8S7jn9a3ToHtw3RJcZiGY21sv5689/9tVu4/+HhsekvWfSEcetutJ5m+0ClynXdLbHkwFncee9Q\n032z3j5cOb1nZokI+P3QcNv2c5t6FHUOxrY1IP0D0Ac8tlnC7+vri4GBgVzjacXqDYOcfflGhoZH\nxqaVS92cf+LCCR8MteqsVi518/Jn9nLZ+sGm5RrFkqWtiSqXunnGvL354e337VQ9pS7Rf/IiBn51\nH5esvatuuS7BB//m0LH3Wuu9lboEguGRbPt0qVv0n7SoYZ3VMXRLDFd/Io3OB9QlRurMbyXO6m17\nzuqNDddPrWXqvacs+9fO7ufWOZLWR0RflrK5dulIeiLwYuCiPNvJS/+aTeMOiKHhEfrXbGprndWG\nhkdYtW5zpnKNYsnS1kQNDY/sdLIHGN4e9K/ZxKp1mxuW2x7s8F5rvbfh7ZE52UOScJvVWR1DvWQP\nsB2aJvuscVZv22brp9YyUH8fbrZ/7ex+blNT3n34FwD/RHIs1CTpTEkDkga2bt2aczit2VKjG6TR\n9J2ps1q9bpBW6tuZODtpy7ahTO+38v20673lUWe7VMYz0f2h3ntqdX3briG3hC/pJcA9EbG+UbmI\nuDAi+iKib/bs2XmFMyFzesotTd+ZOqt1Sztd387E2UlzesqZ3m/l+2nXe8ujznapjGei+0O999Tq\n+rZdQ55n+EcAL5V0J/AF4HmSLsmxvbZbuXQB5VL3DtPKpe6xm27tqrNaudTN8sPnZirXKJYsbU1U\nudTNEQfts9P1lLrEyqULxm7y1tMldnivtd5bqUuUurMlRkj68JvVWR1Dqat+/V1Ad4P5rcRZvW2b\nrZ9ay0D9fbjZ/rWz+7lNTbkl/Ig4OyKeGBHzgVOB70fEirzay8Oyxb2cf+JCenvKiGS0ys7eyKpV\n54ol88a1cd6yhWPl4NEzstHfWWKpbKuWWm0fcdA+487+6sX4uTOezYol88aV75ZYsWRezXmVesol\n+k9Obpqet2whK5bMo1a+nFnq2uGGbb312H/yIvpPWjRunY3G31MujS0/a2Zphxu2tdZXZSw95RIf\n/JtD6T95Uf35pxzKB05exKyZpR2mV6+7yjgb7QOVsY2un8r94IiD9mm6b9bbhyv3L6Xro6dcatt+\nblNT7qN0ACQdDfzjdBulY2Y21bUySif3cfgAEXE1cHUn2jIzs9r8n7ZmZgXhhG9mVhBO+GZmBeGE\nb2ZWEE74ZmYF4YRvZlYQTvhmZgXhhG9mVhBO+GZmBeGEb2ZWEE74ZmYF4YRvZlYQTvhmZgXhhG9m\nVhBO+GZmBeGEb2ZWEE74ZmYF4YRvZlYQTvhmZgXhhG9mVhBO+GZmBeGEb2ZWEE74ZmYF4YRvZlYQ\nTvhmZgXhhG9mVhBO+GZmBeGEb2ZWEE74ZmYF4YRvZlYQM/KqWNLuwDXAY9J2Lo2Id+XVniVWbxik\nf80mtmwbYk5PmZVLF7Bsce+ktNfpWLLENBnl2xX3ZK3PVuO05iZrHeaW8IE/Ac+LiAcllYDrJH0z\nItbm2Gahrd4wyNmXb2RoeASAwW1DnH35RoBcdqZG7QEdjSVLTLXazbt8u+Lu9LadaJzW3GSuw9y6\ndCLxYPqylP5EXu0Z9K/ZNLYTjRoaHqF/zaaOt9fpWLLENBnls2pW72Stz2pTJY7pbDLXYa59+JK6\nJd0A3AN8JyLW1ShzpqQBSQNbt27NM5xd3pZtQy1Nz7O9TsfSrP7Jmp5Vs3ona31mba/TcUxnk7kO\nc034ETGL4jTbAAAIFElEQVQSEYcCTwQOk/T0GmUujIi+iOibPXt2nuHs8ub0lFuanmd7nY6lWf2T\nNT2rZvVO1vrM2l6n45jOJnMddmSUTkRsA64CXtSJ9opq5dIFlEvdO0wrl7pZuXRBx9vrdCxZYpqM\n8lk1q3ey1me1qRLHdDaZ6zDPUTqzgeGI2CapDLwQeH9e7dmjN3w6dfc/S3udHonQ6jrIu3y74u70\ntp1onNbcZK5DReRzH1XSIcBngW6SK4kvRcR7Gi3T19cXAwMDucRjZrYrkrQ+IvqylM3tDD8ibgIW\n51W/mZm1xv9pa2ZWEE74ZmYF4YRvZlYQTvhmZgXhhG9mVhC5DcucCElbgV9Nchj7Ab+b5BiacYzt\nMx3idIztsavGeEBEZHpMwZRK+FOBpIGsY1oni2Nsn+kQp2NsD8foLh0zs8JwwjczKwgn/PEunOwA\nMnCM7TMd4nSM7VH4GN2Hb2ZWED7DNzMrCCd8M7OCKGzCl/QiSZsk3SbpbTXmv1LSTZI2SvqRpEVT\nLcaKcs+S9IikkzoZX9p20xglHS3pBkm3SPrBVItR0t6SvibpxjTG0ychxk9LukfSzXXmS9JH0vdw\nk6RnTMEYp8Ix0zDGinKTdsyk7TeNM5fjJiIK90PyjP7bgQOB3YAbgadWlXkOMCv9+1hg3VSLsaLc\n94ErgZOmWoxAD/AzYF76+nFTMMa3A+9P/54N3Afs1uE4nws8A7i5zvzjgG8CApZ0en/MGOOkHjNZ\nYqzYJyblmGlhXeZy3BT1DP8w4LaIuCMi/gx8AXhZZYGI+FFE3J++XEvyvbxTKsbU/wEuI/mi+E7L\nEuMrgMsj4i6AiOh0nFliDGAvSQL2JEn4j3QyyIi4Jm23npcB/x2JtUCPpCd0JrpEsxinwDGTZT3C\n5B4zQKY4czluiprwe4HNFa/vTqfV81qSs6tOahqjpF7gBODjHYyrUpb1+JfALElXS1ov6bSORZfI\nEuPHgKcAW4CNwP+NiO2dCS+zVvfZyTYZx0xTU+CYySqX4ya3b7zaVUg6hmTnPXKyY6nhAuCfI2J7\ncnI6Jc0Angk8HygDP5a0NiJ+Mblh7WApcAPwPOAg4DuSro2IP0xuWNOTj5m2yOW4KWrCHwTmVrx+\nYjptB+n38l4EHBsR93YotlFZYuwDvpDuuPsBx0l6JCJWdybETDHeDdwbEQ8BD0m6BlgEdCrhZ4nx\ndOB9kXSW3ibpl8DBwE86E2ImmfbZyTbJx0wWk33MZJXLcVPULp2fAk+W9CRJuwGnAl+tLCBpHnA5\n8KpJOhttGmNEPCki5kfEfOBS4O87vOM2jRH4CnCkpBmSZgKHAz+fYjHeRXImhaT9gQXAHR2MMYuv\nAqelo3WWAL+PiF9PdlCVpsAx09QUOGayyuW4KeQZfkQ8IulNwBqSO/afjohbJL0+nf8J4J3AvsB/\npWcDj0QHn7SXMcZJlSXGiPi5pG8BNwHbgYsiouGQuU7HCPwr8BlJG0lGwfxzRHT0MbqSVgFHA/tJ\nuht4F1CqiPFKkpE6twEPk1yVdFSGGCf1mMkY45TQLM68jhs/WsHMrCCK2qVjZlY4TvhmZgXhhG9m\nVhBO+GZmBeGEb2ZWEE74NqVIukrS0qppb5FU91/hJc3P8HTE+ZJeUfG6T9JH0r9fLeljE4j1AkmD\nkpoeR5KulNRTY/q5kv4x/fs9kl6Q/v2WdPy1Wds44dtUs4rkn6MqnZpO3xnzSR5IBUBEDETEmyda\nWZrkTyB5vs1fNSsfEcdFxLYmZd4ZEd9NX74FcMK3tnLCt6nmUuDF6X/FImk+MAe4Nv0v035JN6fP\nXD+leuH0TP5aSdenP89JZ70POCp9vvhZ6bPGv15j+dmSLpP00/TniDpxHg3cQvIQruUVy+8p6eI0\nvpskvTydfqek/dK/3yHpF5KuI/mv3tFlPyPpJElvTt/zVekVz2skXVBR7gxJH8q4Ps3GOOHblBIR\n95E8w+bYdNKpwJfS59ycCBxK8kyRFwD9Gv+I4HuAF0bEM4BTgI+k098GXBsRh0ZEo2T5YeBDEfEs\n4OUkz4WpZTnJVccVJB9QpXT6v5A89mBhRBxC8tz1MZKemb6nQ0n+c/ZZNdbBR0ie3HlMRBwDfAk4\nvqKN04FPN3gPZjUV8tEKNuWNdut8Jf392nT6kcCqiBgBfqvkW4CeRfLv56NKwMckHQqMkDxmthUv\nAJ5a8STFx0raMyIeHJ2QXn0cB/xDRDwgaR3JEze/ni4/1iVV8Xz4UUcBV0TEw2ld1c/1GSciHpT0\nfeAlkn4OlCJiY4vvy8wJ36akrwAfUvI1fjMjYn0Ly54F/JbkKqAL+GOLbXcBSyKi0XJLSb6RaGP6\nwTATGCJJ+Hm5iOSbuW4FLs6xHduFuUvHppz0bPoqkm6Lypu11wKnSOqWNJvka+KqH2G8N/Dr9AtM\nXkXywDSAB4C9MjT/bZJvRAIgvVKothx4XcVTF58EvDAdVfMd4I0Vy8+qWvYaYJmksqS9gOPrxLFD\nvBGxjuTxyK9g529gW0E54dtUtYrkLL0yuV1B0n1zI0nf+D9FxG+qlvsv4G8l3UjyTPuH0uk3ASNK\nvqj8rAbtvhnoS2+4/gx4feXMNKm/CPjG6LT0meXXkSTv80i+qejmNIZjKpePiOuBL6bv4Zskj2+u\n5ULgW5Kuqpj2JeCHNbqJzDLx0zLNpol0VNGHIuJ7kx2LTU8+wzeb4iT1SPoFMORkbzvDZ/hmZgXh\nM3wzs4JwwjczKwgnfDOzgnDCNzMrCCd8M7OC+P9+NJwp32hKugAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x288fb6d14e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(df['volatile acidity'], df['quality'])\n",
    "plt.xlabel('Volatile Acidity')\n",
    "plt.ylabel('Quality')\n",
    "plt.title('挥发性酸度（volatile acidity）与品质（ Quality）',fontproperties=font)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这些图都可以显示出响应变量与解释变量的相关性；让我们构建一个多元线性相关模型表述这些相关性。如何决定哪个变量应该在模型中？哪个可以不在？通过Dataframe.corr()计算两两的关联矩阵（correlation matrix）。关联矩阵进一步确认了酒精度与品质的正相关性，挥发性酸度与品质的负相关性。挥发性酸度越高，酒喝着就越向醋。总之，我们就假设好酒应该具有酒精度高、挥发性酸度低的特点，虽然这和品酒师的味觉可能不太一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 拟合与评估模型\n",
    "\n",
    "现在，我们把数据分成训练集和测试集，训练回归模型然后评估预测结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "R-squared: 0.379323764551\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\lymanZHANG\\Anaconda3\\lib\\site-packages\\sklearn\\cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pylab as plt\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.cross_validation import train_test_split\n",
    "df = pd.read_csv('mlslpic/winequality-red.csv', sep=';')\n",
    "X = df[list(df.columns)[:-1]]\n",
    "y = df['quality']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)\n",
    "regressor = LinearRegression()\n",
    "regressor.fit(X_train, y_train)\n",
    "y_predictions = regressor.predict(X_test)\n",
    "print('R-squared:', regressor.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开始和前面类似，加载数据，然后通过train_test_split把数据集分成训练集和测试集。两个分区的数据比例都可以通过参数设置。默认情况下，25%的数据被分配给测试集。最后，我们训练模型并用测试集测试。\n",
    "\n",
    "R方值0.38表明38%的测试集数据都通过了测试。如果剩下的72%的数据被分到训练集，那效果就不一样了。我们可以用交叉检验的方法来实现一个更完善的效果评估。上一章我们介绍过这类方法，可以用来减少不同训练和测试数据集带来的变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.290041628842 [ 0.13200871  0.31858135  0.34955348  0.369145    0.2809196 ]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "regressor = LinearRegression()\n",
    "scores = cross_val_score(regressor, X, y, cv=5)\n",
    "print(scores.mean(), scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里cross_val_score函数可以帮助我们轻松实现交叉检验功能。cv参数将数据集分成了5份。每个分区都会轮流作为测试集使用。cross_val_score函数返回模拟器score方法的结果。R方结果是在0.13到0.37之间，均值0.29，是模拟器模拟出的结果，相比单个训练/测试集的效果要好。\n",
    "\n",
    "让我们看看一些模型的预测品质与实际品质的图象显示结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEXCAYAAABGeIg9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2UXFWZ7/HvL00jnciQBMJbkwDKGC9chEAuBBIRxSFy\nBckAzhJ1uDhqboRh5g4YBW9mRIcBZkXvUvReeRFFB4iDvLQISJDBl8CsRBMaaJFEECGhQQkvAYmt\nCZ3n/nFOQaWrqtOVrlPVdc7vs1avrnrqbZ+kup46ez97b0UEZmZm5ca1ugFmZjb2ODmYmVkFJwcz\nM6vg5GBmZhWcHMzMrIKTg5mZVdih1Q0wq5ekHan8YrMFUET8qcGvJaATGA9MAHYBJgF7AI9ExCPb\n+bx7AJsj4oVGtXUbr5fJcVh+OTnYmCfpCuAgYEdgInA7cDBwAPA88CIQ6X1vABYCzwK7Av8KvAy8\nAMyKiEslLQDOTe9TbhowPyLuTJ/rUeC3wB/TnwHg9+nrvQj8rqyNqyLi8DoO61jg3cDHhxzrmcA/\npldvBN5R5bEfjYiHszgOsxInB2sH55J8kJ4I3BIRdwBI+g6wMCLWpddPAA4DLgaeAXYG3gicAXxp\nyHNeHBHXlAckXTjkPs9HxNvLbv8McFdErBzyuA7gLZKW12j/g0Af8AmSD+WSzrLHTACuI/kQ/2eS\nZHh3RHy6xnM2/DjMyjk52JgXERslnQx0RsQdks4A/oaka+krkqYAPwCuIPmQfSPwD8DXR/vakr4C\nzEiv7gd8QNLL6fUbIuIyoBv4aUS8d5jn+TjwSWAv4NGIWCbpeOCZiOiT9I6y1wGYDSwabfvrPA6z\n1zg52Jgn6bvAPsDeku4FfgR8gSQRPA28maTv/Ebg/47gKTcDn5L0sSHxPYF7y65HRJxT1o4vAD0R\nce+Qx00BZkmq9U18dkRcJakTmAx8SdKh6eNWAn0R8RPgJ2m30htJxlDOTZMiwIHAL9PL90fEWRkc\nh9lrnBxszIuI90s6kWSM4e3Ac+lNewKvpJd/DRwPnJRe7xjm+a4Grh7Ja0s6ALgPeDQNzUrGdpkc\nEQemz7eKZHyj2uPvLxskfz/w18CFwCpgMVt3M5W8AjwELAOuSu/zHxExR9IuEfFSFsdhVs6lrNZu\nfgKUPmz3Bl4qu+0ykmqcDmq8tyV9WNKvJN2b/j5T0iJJfWnsSUnHDnnYDyJiDnANyUDvHJIB7rpE\nxPXAecBBEfEqsD+wtsbdvwnMA74MvCVt+1SScYmWHocVg88crK1ExGVplU4HcArJN+cD0pv/G0k/\n/VySJPEzYBPJh2y5iyPimrQLp+SciPixpEUAknansgroUODn5QFJtwBTazT3QkDp/S4E3lP2uE+S\nJLfr0m/wAPenbT6UpKtsIjApHZMgItZJ6pc0p9HHYTaUk4O1i07SclVgCUl3S2kQ+gbgN8AKkqTx\nKeB64N9JBq5PAd5V9vhOSTulz7m5xusdBfSSfLO+VdKuwBxgH0lfBL4FEBF/WavBkqaTVB8REReS\nJAskHUiS1H4P3AZcHhHPpbedSXJmtJokeVxadvyk1/8ZuLORx2E2lLuVbMyTdDXwReAhSUeS9MXf\nFhF/R/JtfDpwC0m56F3AeRFxKfAR4ATgJuBK4IH0Kc8j+XA9r+xlvijpbuDM9Pr89Llmk4xz3Aos\nAM4GTgM+KOndVdr6OUlrJD1EMh/jsjT+JkkfkfR9krLas4CjgceBOyRdkSYTSCalXZfe582S+kjm\nIxARvyEZt8j0OMzkzX6snUgaB7whIgaGxDsiYlDShIjY2IDXOYok2ZxNMpB7X5T9sUjaF5gWEcuq\nPFYkf1tbymIfIul+6omI1VXufzLwx9LEtSG3jyt/rmYdhxWbk4OZmVVwt5KZmVVwcjAzswptW620\n2267xX777dfqZpiZtZVVq1Y9FxFTtnW/tk0O++23HytXet0wM7N6SHpyJPdzt5KZmVVwcjAzswpO\nDmZmVsHJwczMKjg5mJlZBScHMzOr4ORgZmYV2naeg5kVW09vP4uXruHpDQPsPbGLhXOnM29Gd6ub\nlRtODmbWdnp6+7ng5j4GNg8C0L9hgAtu7gNwgmgQdyuZWdtZvHTNa4mhZGDzIIuXrmlRi/LHycHM\n2s7TGwbqilv9nBzMrO3sPbGrrrjVz8nBzNrOwrnT6ers2CrW1dnBwrnTazzC6uUBaTNrO6VBZ1cr\nZcfJwcza0rwZ3U4GGXJyMLO25HkO2XJyMLO243kO2fOAtJm1Hc9zyJ6Tg5m1Hc9zyJ6Tg5m1Hc9z\nyJ6Tg5m1Hc9zyJ4HpM2s7XieQ/acHMxyoIhlnZ7nkC0nB7M257JOy4LHHMzanMs6LQs+czBrc0Ut\n61zU08eSFesYjKBD4vQjp3LRvINb3azc8JmDWZsrYlnnop4+rl2+lsEIAAYjuHb5Whb19LW4Zfnh\n5GDW5t751il1xfNgyYp1dcWtfk4OZm3uR6vX1xXPg9IZw0jjVj8nB7M2V9QxB8uWk4NZmyvimIPq\njFv9nBzM2lwRl5Ko1XmU506lnt5+Zl96D/uffzuzL72Hnt7+TF/PycGszc2b0c2ph3fToeR7c4fE\nqYfne/Zw6VhHGm93pYmO/RsGCF6f6JhlgnBysNxp9jesVuvp7ef6IWWd1y9fm+vjnvWmSXXF210r\nJjo6OViutOIbVqtdcPNDbBkS25LG8+qJ56sPtteKt7tWFB00JTlImijpRkmrJT0i6aghtx8r6SVJ\nD6Q//9SMdln+FHEpiYHNQ1PD8PE8KFqFViuKDpp15vBl4M6IeCtwCPBIlfssi4hD05/PN6ldljNF\n+9Aoql26OuuKt7tWFB1kvraSpF2AY4AzASJiE7Ap69e1Ytp7Yhf9VRJBnss6i6jWuHNOx6Nbsn9F\nMxbe2x9YD3xT0iHAKuDvI2LjkPsdLekhoB/4ZEQ8PPSJJM0H5gNMmzYt21ZbW1o4d/pWy1dD/ss6\ni+jFP2yuK54Hzd6/ohndSjsAhwFfi4gZwEbg/CH3uR+YFhFvA74C9FR7ooi4MiJmRsTMKVPyu26M\nbb8ilnUWUdFKWVuhGcnhKeCpiFiRXr+RJFm8JiJejohX0st3AJ2SdmtC2yxnenr7uWlV/1ZlnTet\n6s91tVIRPyi9tlL2Mk8OEfFbYJ2k0nn9ccAvy+8jaU8peSdLOiJt1/NZt83yp4jVSkX8oBxXI+/V\nilv9mrXZzznAdZJ2BB4HPiJpAUBEXA6cBnxC0qvAAPCBiBy/sy0zRaxW6q4xCN+d40H4LTU+HWrF\nrX5NSQ4R8QAwc0j48rLbvwp8tRltsXwrYrWSB+EtC54hbblSxEXo5s3o5pJTDqZ7YhciOWO45JSD\nPQhvo+I9pC1XWlEPPhY0u8zR8s/JwXLHH5SWRz29/bmbBGdmGWv2B4c1V2lBydK4UmlBSSCz/2eP\nOZi1uSKuRFu0neC8ZLeZ1a2IczsO2H1CXfF214oSbXcrWe4UrYulWunucPE8eOzZoUuzDR9vd60o\n0faZg+VKEbtYijhbuGh7SLeiRNvJwXKliF0sni2cf62Yy+JuJcuVIi6fYcWQxyW7zZpm4vjqO4HV\niptZdU4Oliu1lmv0Mo75UsRlypvNycFyZcNA9Z3AasWtPZ1+5NS64lY/JwfLFX+jNGsMD0hbrhRx\n45siun7F2prxi+Yd3OTWNMeinj6WrFjHYAQdEqcfOTXTY/WZg+VKrQ1u8rzxTREVrXx3UU8f1y5f\nu9X2t9cuX8uinr7MXtPJwXJl4dzpdA6Z/dU5Trnez8Hyb8mKdXXFG8HJwfJn6PCChxtyp7PGJ1et\neLtrRXdpTv8pragWL13D5sGt/2A2D0auZ0gX0Rt3qj5vpVbc6ufkYLniGdLFsOEPNUqWa8Stfk4O\nliu1VqnMcvXKViti+W7RZsK3otDCycFypRWrV7ZaESeEFW0mvFdlNRulVqxe2Woz951cVzwPijYT\n3quymjVAs1evbLULb324ZrxI/w5551VZzawuRfsWbc3h5GBmZhWcHMzMrIKTg5m1naLNkG4F/1Oa\nWdvZvKW+uNXPycHMzCo4OZiZWQXPcyiAnt5+Fi9dw9MbBth7YhcL507Pdf170Y7XLAtODjnX09vP\nBTf3MbB5EID+DQNccHOyQUgePzCLdrwAE3bsYOOmwapxs+3lbqWcW7x0zWsflCUDmwdzu4R10Y4X\nYNOrlYlhuLjZSDg55FzRlrAu2vGCK3csG04OOVe0JayLdrxmWWlKcpA0UdKNklZLekTSUUNul6TL\nJD0m6SFJhzWjXUVQtCWsvYd0MdTaqiLHW1g0XbPOHL4M3BkRbwUOAR4ZcvsJwJ+nP/OBrzWpXblX\nxCWsi7aHdEeN46sVz4MPHTmtrrjVL/NqJUm7AMcAZwJExCZg05C7nQx8OyICWJ6eaewVEc9k3b4i\nKNIS1sPtIZ3Xf4PBGhvc1Irnwcx9J3Pt8rVV49YYzThz2B9YD3xTUq+kr0uaMOQ+3cC6sutPpTGz\nuhRxQLqIhtvDwhqjGclhB+Aw4GsRMQPYCJy/PU8kab6klZJWrl+/vpFttJzwgHQxeA+L7DUjOTwF\nPBURK9LrN5Iki3L9QPmGt/uksa1ExJURMTMiZk6ZMiWTxlp784C0WWNknhwi4rfAOkmlv87jgF8O\nudutwBlp1dIs4CWPN9j2Glre73L//Jk0vrOuuNWvWdVK5wDXSXoIOBS4WNICSQvS2+8AHgceA64C\nzmpSuyxnPvf9hxncsvVI7OCW4HPfd190nnz2pIPoHFKO1dkhPnvSQS1qUf40ZW2liHgAmDkkfHnZ\n7QGc3Yy2FNGinj6WrFjHYAQdEqcfOZWL5h3c6mZl4sU/VO9zrhW39lSqPPMCi9nxwns5t6inb6uS\nv8GI167nNUFYMRSpRLsVhu1WkjS9NJtZ0t/WuM+7JM3OonE2ektWrKsrbtYuenr7mX3pPex//u3M\nvvQeenoralhsFLY15nAgcGx6+a8AJD0s6afpz8eA/w1UzkaxMWEwqs+EqhU3awelpdn7NwwQvL40\nuxNE42wrOfwOmDQk9kJEHBMRxwCrgb6I8NfQMaqjxmIzteJm7aCIS7M327aSw4vApHQhvD+TdDQQ\nkt4q6WDgXOBfs26kbb/Tj5xaV9ysHXgmfPa2NSD9EvBnwFxgV+Av0/hskpLU/UkmrHlOwhhVGnQu\nSrWSFcPE8Z1VK9Amep5Dw2wrOWwEdoqISySdEBELJf00Iq4GkDQZ+B7w9qwbatvvonkHOxlYrtQa\nMvNQWuNsKzn8AdgpvVzqpH6LpG+kl28Ffi7p6Ij4zywaaGY21Es11lCqFbf6DZscImKzpFJyuD/9\nPRso5ecXgVfTmJPDGNXT2+/JQpYre0/sor/K+IIXWGyckUyC20vSf5Js2HZfWfxZoBe4C/h/WTTO\nRq9U8leq7CiV/AFOENa29tu1enLYb1cnh0bZZnKIiLdUi0uaAhxFMs/hImBFtftZaw1X8pfH5DBO\nsKVKv/M4V+7myvLHX6wrbvXbroX3JB0D/BdgA7C4bDluG2OKVvJ31Juq7wRWK27tyZM7s7fNMwdJ\nx5ddfTUi7gG+DVxJMkj9P4CqZxfWekXrm33i+epJr1bc2lOHVDUReHJn44zkzOEqYBZJF1JpJdV1\nEXFxRPwL8GRWjbPRWzh3Ol2dHVvFujo7crv5TdHOlIpq1puGLtwwfNzqN5IB6bUR8XkAScelsV3S\nriUBE7NqnI1e0ZY2LtqZUlH5DDF7I0kO1Trx/h2YQ5IcehraImu4Ii1tPH7H6ifDteLWnnyGmL2R\nJIdqnXhPkSQNUT15mLXEo89urCtu7Wn8jh1s3DRYNW6NMZKvU9U+/D+T/u4Ezm9cc8zMtu0PVRLD\ncHGr30jOHPaQdAbJWUKpHvDZiPi2pDcAH8ysdWZmVdTqrnA3RuOMJDn8H6Cr7DLA7mnC2BHYM4uG\nWeN4+QzLG5eyZm8kM6SvqBL+PEmSHgT+pdGNssbx8hmWR6cfOXWrvdHL49YYI5kE92kqz9ZK6XkL\n8ECjG2WNU7TlM6wYZu47mSU/W8dg2VopHePEzH09E75RRtKttJzaXXmdwBXAAQ1rkTVU0Ur+apXP\nubMhXxYvXbNVYgAY3BL+0tNAI0kOk4Ez2PpvrvQ3eCrwUAbtsgYp2qQwD1QWQ9G+9LTCSJLDzlSf\n6HYnyZpKyxraImuohXOnbzXmAPlePsOKoWhfelphJPMcPg38vspPJ/AN4PbMWmejNm9GN6ce3v1a\nFUeHxKmHF2fGtOVT0dYMa4WRnDm8QJIMziU5O+9If84DvhcRv8queTZaPb393LSq/7Wyv8EIblrV\nz8x9JztBWNsq2pphrTCS5LA7sBfwnbJYB7AJWChpZbqMt41BRatW8mY/xVGkNcNaYSTJYTHQDfyK\n1weiu4AfAD8Elko6KiI8EjQGFW3grlpiGC5u7cuTO7M17JiDpB2B9wP3Az8BzgZuJVmRdSnwT8Dl\nTgxjV60BOg/cWTsrTe7s3zBA8Prkzp7e/lY3LTe2NSAdABGxFJhPcrZwEskZwyUkS3dfJHnO+ljl\ngTvLo+G6S60xttWtNI7X5w/9d5KtQXcuu3034N8ivHHrWOWBO8ujonWXtsJIxhwmSeoG1gO/BnYB\nNqTXLwfek13zrBE8cGd543kO2RvJPIfdSAalJwF7kyyV8T7gNqAvIjZn1zwzs0ruLs3eSJLDYxHx\nQZLxh40kZw/LgIXAXpI+lGH7zMwqzJvRzSWnHEz3xC4EdE/s4pJTDvYZcgNtq1tpC6+vnfQN4Ejg\nOeBPaezjwDJJN0bEn6o83swsE+4uzdawZw5pl9GJkuZGxL8B7wKOAXaMiCsj4mXg404MZmb5MpIB\n6Q3AKZI+COwDfBfokPR3JF1N21yVVdITJEtwDAKvRsTMIbcfC3wP+E0aujkiPj/CY7Bt8GQhyyO/\nr7M1kuSwiWRxvZOBp4HHeL28tRO4mpHt5/DOiHhumNuXRcSJI3geq0PRdoLzfg7FULT3dSuMZED6\nYeCHEfFRoD8i7oiI29OfHuDHmbbQRqVok4W8n0MxFO193QrbTA4RsaC0PEZE/FWV2z82gtcJ4G5J\nqyTNr3GfoyU9JOkHkg6qdgdJ8yWtlLRy/fr1I3hZq1YLPlzcrB14Elz2RnLm0AhzIuJQ4ATgbEnH\nDLn9fmBaRLwN+ArVNxciHQSfGREzp0yZkm2LzWzM8pph2WtKcoiI/vT3s8AtwBFDbn85Il5JL98B\ndErarRltM7P240lw2cs8OUiaIGnn0mXgeOAXQ+6zZ2nxPklHpO16Puu2FUFHjTURa8XN2oEnwWVv\nJNVKo7UHcEv62b8DcH1E3ClpAUBEXA6cBnxC0qvAAPABL+bXGKcfOZVrl6+tGjdrZ54El63Mk0NE\nPA4cUiV+ednlrwJfzbotRTRz38ks+dk6Bst2u+kYJ2buO7mFrTKzsa5ZA9LWIouXrtkqMQAMbgmX\n/JnZsJwccs4lf2a2PZwccs4lf2a2PZwccs4lf2a2PZpRrWQt5G1C869DMFiltq/D1co2Ck4OBeCS\nv3yrlhiGi5uNhLuVzMysgpODmZlVcHIwM7MKTg5mZlbBycHMzCo4OZiZWQUnBzMzq+DkYGZmFZwc\nzMysgpODWZv78KxpdcXNRsLLZxRAT2+/11Yys7r4zCHnenr7WfjdB+nfMEAA/RsGWPjdB+np7W91\n06xBrl9RuQ3scHGzkXByyLkLb32YzUN2gtu8Jbjw1odb1CJrtC01FtirFTcbCSeHnNswsLmuuJkZ\nODmYmVkVTg5mZlbBySHnJo3vrCtuZgZODrn33rftVVfczAwKmBx6evuZfek97H/+7cy+9J7cl3Te\n9uAzdcWt/YzvrP5nXCtuNhKFmgTX09vPBTf3MbB5EEhq/i+4uQ8gt5PCXK2Uf5LqipuNRKG+Wixe\nuua1xFAysHmQxUvXtKhFZqO3cdNgXXGzkShUcnh6w0Bd8TzwgLSZbY9CJYe9J3bVFc+Dz550EJ0d\nW3cvdHaIz550UItaZI3WVWNsoVbcbCQK9e5ZOHc6XZ0dW8W6OjtYOHd6i1qUvXkzull82iF0T+xC\nQPfELhafdkhux1iKaKch7+ltxc1GolAD0qUPxKKtUDpvRnfuj7HINvyhRtFBjbjZSBQqOYA/KC1/\n9p7YRX+VcbM8d5da9grVrWSWR0XsLrXsFe7MwRvfWN7Mm9HNyidfYMmKdQxG0CFx6uE+Q7bRKdSZ\nQ2kSXPnGNxfc3Jf7WdKWbz29/dy0qp/BSDZwGIzgplX9fl/bqBQqORR1ElzRlgwpmqK+ry1bhepW\nKuIkuCIuGVI0RXxfW/aacuYg6QlJfZIekLSyyu2SdJmkxyQ9JOmwLNpRxElw/laZf0V8X1v2mtmt\n9M6IODQiZla57QTgz9Of+cDXsmhAEas6ivatstYbOs/9p0V8X1v2xkq30snAtyMigOWSJkraKyIa\nuq50ESfBFa0Gfkud8Two4vvastes5BDA3ZIGgSsi4soht3cD68quP5XGtkoOkuaTnFkwbdq07WpI\n0SbBvfOtU7h2+dqqcTOzWpqVHOZERL+k3YEfSlodET+t90nSpHIlwMyZM6PRjcyjH61eX1fc2o+L\nDiwLTemKjYj+9PezwC3AEUPu0g9MLbu+TxqzUfKYw/DxPHDRgWUh878ZSRMk7Vy6DBwP/GLI3W4F\nzkirlmYBLzV6vKGoilbJ0rVj9ZVIa8XzoGhfAKw5mvGFag/gXkkPAj8Dbo+IOyUtkLQgvc8dwOPA\nY8BVwFlNaFch1BpbyOuYQxF3RSvaFwBrjszHHCLiceCQKvHLyy4HcHbWbSkijznk38K507cacwCX\nstrojZVSVsuIuxzyz6WslgUnh5wr2jyHoipaibZlL89FHIZnz5rZ9vGZQ84VrctBgqgyA0ZqflvM\n2pmTQwEUqcuhWmIYLm5m1blbyXKlu8ZYSq24mVXn5GC54jEWs8Zwt5LlStHGWMyy4uRguVOkMRaz\nrLhbyczMKjg5mJlZBScHMzOr4ORgZmYVnBzMzKyCq5UKoKe336WdZlYXJ4ec8/7CZrY93K2Uc95f\n2My2h5NDznmzHzPbHk4OOef9hc1sezg55JwXojOz7eEB6ZzzQnRmtj2cHAqgaAvRuXTXbPScHCxX\nXLpr1hgec7BccemuWWM4OViuuHTXrDGcHCxXXLpr1hhODpYrLt01awwPSFuuuHTXrDGcHCx3ila6\na5YFdyuZmVkFJwczM6vg5GBmZhWcHMzMrIKTg5mZVXByMDOzCoqIVrdhu0haDzw5iqfYDXiuQc1p\nF0U75qIdL/iYi2C0x7tvREzZ1p3aNjmMlqSVETGz1e1opqIdc9GOF3zMRdCs43W3kpmZVXByMDOz\nCkVODle2ugEtULRjLtrxgo+5CJpyvIUdczAzs9qKfOZgZmY1ODmYmVmFwiUHSTtJ+pmkByU9LOlz\nrW5TM0jqkNQr6bZWt6UZJD0hqU/SA5JWtro9WZM0UdKNklZLekTSUa1uU5YkTU//b0s/L0v6X61u\nV9Yk/UP6ufULSUsk7ZTZaxVtzEGSgAkR8YqkTuBe4O8jYnmLm5YpSecCM4E/i4gTW92erEl6ApgZ\nEYWYHCXpW8CyiPi6pB2B8RGxodXtagZJHUA/cGREjGZi7JgmqZvk8+rAiBiQdANwR0Rck8XrFe7M\nIRKvpFc7059cZ0hJ+wDvBb7e6rZY40naBTgGuBogIjYVJTGkjgN+nefEUGYHoEvSDsB44OmsXqhw\nyQFe62J5AHgW+GFErGh1mzL2JeBTwJZWN6SJArhb0ipJ81vdmIztD6wHvpl2HX5d0oRWN6qJPgAs\naXUjshYR/cAXgLXAM8BLEXFXVq9XyOQQEYMRcSiwD3CEpP/a6jZlRdKJwLMRsarVbWmyOen/8QnA\n2ZKOaXWDMrQDcBjwtYiYAWwEzm9tk5oj7UJ7H/DdVrcla5ImASeTfBnYG5gg6cNZvV4hk0NJeur9\nI+A9rW5LhmYD70v74L8DvEvSta1tUvbSb1lExLPALcARrW1Rpp4Cnio7A76RJFkUwQnA/RHxu1Y3\npAneDfwmItZHxGbgZuDorF6scMlB0hRJE9PLXcBfAKtb26rsRMQFEbFPROxHcvp9T0Rk9m1jLJA0\nQdLOpcvA8cAvWtuq7ETEb4F1kqanoeOAX7awSc10OgXoUkqtBWZJGp8W1hwHPJLVi+2Q1ROPYXsB\n30orHMYBN0REIco7C2QP4Jbk74cdgOsj4s7WNilz5wDXpd0sjwMfaXF7Mpcm/r8A/mer29IMEbFC\n0o3A/cCrQC8ZLqVRuFJWMzPbtsJ1K5mZ2bY5OZiZWQUnBzMzq+DkYGZmFYpYrWS2TZL2ADZHxAtN\neC2RLOMyHpgA7AJMIqm6eiQiMitXNKvFycGsumNJJh19vBSQdCbwj+nVG4F3VHncRyPiYUkLgHNJ\nlmgpNw2YXyqtlfQo8Fvgj+nPAPB74MX0pwiTu2wMcimrGSDpb4FPkHwwl3QCm9PLE4DrSD7IDwLu\niogfDvN8C4A/Dl0xU9KFwPKy5LA8ImaV3f6Z9Llzv8y4jW0+czBL/An4JMkkyUcjYpmk44FnIqJP\n0juAGel9ZwOLGvXCkr5S9tz7AR+Q9HJ6/YaIuKxRr2U2Uk4OZkBEXJXu7zEZ+JKkQ4EpwEqgLyJ+\nIml/4I0kq9ueK+nk9OEH8vpyFfdHxFkkZxyfkvSxIS+1J8ma/GUvHeeUrkj6AtATEfdi1kJODmav\nez/w18CFwCpgMVt3MwG8AjwELAOuSm//j4iYI2mXiHgJICKuJt1fYVskHQDcBzyahmalS39MjogD\nR3NAZtvLpaxmqYi4HjgPOCgiXiVZGnltlbt+E5gHfBl4C4CkqSRjEkj6sKRfSbo3/X2mpEXptqX3\nSnpS0rFDnvMHETEHuIZkwHoOkHmllFktPnMw47WB4veUXf8kyZr516Xf4gHeRrLQ2ZuBicCkdDyC\niFgnqV/SnPS+F0fENWmFU8k5EfFjSYvS19idymqmQ4GfN/bozOrnMwczICIujIhZaeXQ3wAvAWuA\n24AT0/ikG9omAAAA80lEQVRZJAPXqwEBl6YP70x/XwosKMXSzd9Lt1VzFMnKmi8At0raFZgDfE7S\n24FvNer4zOrl5GAGSHqTpI9I+j7JtqpnkWyk8jhwh6QrgDeQTEq7Lr39zZL6SOYjEBG/IRmzgKR7\n6s70d8kXJd0NnJlenw/cRVL99HbgVpLkcjZwGvBBSe/O5ojNhud5DmaApA8BU0kqhVYPuU0k2zP+\ncei+EJLGRcR27c0t6SigjyQZ3AfcF2V/kJL2BaZFxLLteX6z0XByMDOzCu5WMjOzCk4OZmZWwcnB\nzMwqODmYmVkFJwczM6vg5GBmZhWcHMzMrML/B4XNAHRzdS5/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x288fb65e4e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(y_test, y_predictions)\n",
    "plt.xlabel('实际品质',fontproperties=font)\n",
    "plt.ylabel('预测品质',fontproperties=font)\n",
    "plt.title('预测品质与实际品质',fontproperties=font)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "和假设一致，预测品质很少和实际品质完全一致。由于绝大多数训练数据都是一般品质的酒，所以这个模型更适合预测一般质量的酒。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 梯度下降法拟合模型\n",
    "\n",
    "前面的内容都是通过最小化成本函数来计算参数的：\n",
    "\n",
    "β = (X<sup>T</sup>X)<sup>-1</sup>X<sup>T</sup>Y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里X是解释变量矩阵，当变量很多（上万个）的时候， X<sup>T</sup>X计算量会非常大。另外，如果X<sup>T</sup>X的行列式为0，即奇异矩阵，那么就无法求逆矩阵了。这里我们介绍另一种参数估计的方法，梯度下降法（gradient descent）。拟合的目标并没有变，我们还是用成本函数最小化来进行参数估计。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "梯度下降法被比喻成一种方法，一个人蒙着眼睛去找从山坡到溪谷最深处的路。他看不到地形图，所以只能沿着最陡峭的方向一步一步往前走。每一步的大小与地势陡峭的程度成正比。如果地势很陡峭，他就走一大步，因为他相信他仍在高出，还没有错过溪谷的最低点。如果地势比较平坦，他就走一小步。这时如果再走大步，可能会与最低点失之交臂。如果真那样，他就需要改变方向，重新朝着溪谷的最低点前进。他就这样一步一步的走啊走，直到有一个点走不动了，因为路是平的了，于是他卸下眼罩，已经到了谷底深处，小龙女在等他。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通常，梯度下降算法是用来评估函数的局部最小值的。我们前面用的成本函数如下：\n",
    "\n",
    "$ SS_{res} = \\sum_{i=1}^{n}{(y_i - f(x_i))^2} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以用梯度下降法来找出成本函数最小的模型参数值。梯度下降法会在每一步走完后，计算对应位置的导数，然后沿着梯度（变化最快的方向）相反的方向前进。总是垂直于等高线。\n",
    "\n",
    "需要注意的是，梯度下降法来找出成本函数的局部最小值。一个三维凸（convex）函数所有点构成的图形像一个碗。碗底就是唯一局部最小值。非凸函数可能有若干个局部最小值，也就是说整个图形看着像是有多个波峰和波谷。梯度下降法只能保证找到的是局部最小值，并非全局最小值。残差平方和构成的成本函数是凸函数，所以梯度下降法可以找到全局最小值。\n",
    "\n",
    "梯度下降法的一个重要超参数是步长（learning rate），用来控制蒙眼人步子的大小，就是下降幅度。如果步长足够小，那么成本函数每次迭代都会缩小，直到梯度下降法找到了最优参数为止。但是，步长缩小的过程中，计算的时间就会不断增加。如果步长太大，这个人可能会重复越过谷底，也就是梯度下降法可能在最优值附近摇摆不定。\n",
    "\n",
    "如果按照每次迭代后用于更新模型参数的训练样本数量划分，有两种梯度下降法。批量梯度下降（Batch gradient descent）每次迭代都用所有训练样本。随机梯度下降（Stochastic gradient descent，SGD）每次迭代都用一个训练样本，这个训练样本是随机选择的。当训练样本较多的时候，随机梯度下降法比批量梯度下降法更快找到最优参数。批量梯度下降法一个训练集只能产生一个结果。而SGD每次运行都会产生不同的结果。SGD也可能找不到最小值，因为升级权重的时候只用一个训练样本。它的近似值通常足够接近最小值，尤其是处理残差平方和这类凸函数的时候。\n",
    "\n",
    "下面我们用scikit-learn的SGDRegressor类来计算模型参数。它可以通过优化不同的成本函数来拟合线性模型，默认成本函数为残差平方和。本例中，我们用波士顿住房数据的13个解释变量来预测房屋价格："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import load_boston\n",
    "from sklearn.linear_model import SGDRegressor\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.cross_validation import train_test_split\n",
    "data = load_boston()\n",
    "X_train, X_test, y_train, y_test = train_test_split(data.data, data.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "scikit-learn加载数据集的方法很简单。首先我们用train_test_split分割训练集和测试集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "operands could not be broadcast together with shapes (1,1651) (4927,) (1,1651) ",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-24-97507bc3e43a>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mX_train\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mX_scaler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit_transform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;31m#(X_train)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0my_train\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0my_scaler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit_transform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my_train\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;31m#(y_train)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mX_test\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mX_scaler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtransform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_test\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;31m#(X_test)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0my_test\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0my_scaler\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtransform\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0my_test\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;31m#(y_test)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[1;31m#np.array(xpredict).reshape(1,-1)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[0;32mC:\\Users\\lymanZHANG\\Anaconda3\\lib\\site-packages\\sklearn\\preprocessing\\data.py\u001b[0m in \u001b[0;36mtransform\u001b[0;34m(self, X, y, copy)\u001b[0m\n\u001b[1;32m    658\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m    659\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mwith_mean\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m--> 660\u001b[0;31m                 \u001b[0mX\u001b[0m \u001b[1;33m-=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmean_\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    661\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mwith_std\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m    662\u001b[0m                 \u001b[0mX\u001b[0m \u001b[1;33m/=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mscale_\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: operands could not be broadcast together with shapes (1,1651) (4927,) (1,1651) "
     ]
    }
   ],
   "source": [
    "X_scaler = StandardScaler()\n",
    "y_scaler = StandardScaler()\n",
    "X_train = X_scaler.fit_transform(np.array(X_train).reshape(1,-1))#(X_train)\n",
    "y_train = y_scaler.fit_transform(np.array(y_train).reshape(1,-1))#(y_train)\n",
    "X_test = X_scaler.transform(np.array(X_test).reshape(1,-1))#(X_test)\n",
    "y_test = y_scaler.transform(np.array(y_test).reshape(1,-1))#(y_test)\n",
    "#np.array(xpredict).reshape(1,-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后用StandardScaler做归一化处理，后面会介绍。最后我们用交叉验证方法完成训练和测试："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "regressor = SGDRegressor(loss='squared_loss')\n",
    "scores = cross_val_score(regressor, X_train, y_train, cv=5)\n",
    "print('交叉验证R方值:', scores)\n",
    "print('交叉验证R方均值:', np.mean(scores))\n",
    "regressor.fit_transform(X_train, y_train)\n",
    "print('测试集R方值:', regressor.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结\n",
    "\n",
    "本章我们介绍了三类线性回归模型。首先，通过匹萨价格预测的例子介绍了一元线性回归，一个解释变量和一个响应变量的线性拟合。然后，我们讨论了多元线性回归，具有更一般形式的若干解释变量和一个响应变量的问题。最后，我们讨论了多项式回归，一种特殊的多元线性模型，体系了解释变量和响应变量的非线性特征。这三种模型都是一般线性模型的具体形式，在第4章，从线性回归到逻辑回归（Logistic Regression）。\n",
    "\n",
    "我们将残差平方差最小化为目标来估计模型参数。首先，通过解析方法求解，我们介绍了梯度下降法，一种可以有效估计带许多解释变量的模型参数的优化方法。这章里的案例都很简单，很容易建模。下一章，我们介绍处理不同类型的解释变量的方法，包括分类数据、文字、图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 2 70 90  1]]\n"
     ]
    }
   ],
   "source": [
    "#This makes it into a 2d array\n",
    "temp =  [2 ,70 ,90 ,1] #an instance\n",
    "temp = np.array(temp).reshape((1, -1))\n",
    "print(temp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
